-- Logs begin at Mon 2022-09-26 05:02:40 UTC, end at Mon 2022-09-26 06:17:36 UTC. -- Sep 26 05:28:17.288798 dv-cinder-836083-14-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 26 05:28:17.298686 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 26 05:28:17.300754 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 26 05:28:17 2022] *** Sep 26 05:28:17.300754 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 26 05:28:17.300754 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 26 05:28:17.300754 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: nodename: dv-cinder-836083-14-iscsi Sep 26 05:28:17.300754 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: machine: x86_64 Sep 26 05:28:17.300754 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: clock source: unix Sep 26 05:28:17.300754 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: pcre jit disabled Sep 26 05:28:17.300754 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: detected number of CPU cores: 4 Sep 26 05:28:17.300754 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: current working directory: / Sep 26 05:28:17.300754 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: detected binary path: /usr/bin/uwsgi-core Sep 26 05:28:17.300754 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: your processes number limit is 47680 Sep 26 05:28:17.300754 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: your memory page size is 4096 bytes Sep 26 05:28:17.300754 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: detected max file descriptor number: 2048 Sep 26 05:28:17.300754 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: lock engine: pthread robust mutexes Sep 26 05:28:17.301921 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: thunder lock: enabled Sep 26 05:28:17.301921 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 26 05:28:17.301921 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 26 05:28:17.343489 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: Python main interpreter initialized at 0x55dff948b9c0 Sep 26 05:28:17.343489 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: python threads support enabled Sep 26 05:28:17.343489 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: your server socket listen backlog is limited to 100 connections Sep 26 05:28:17.343489 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: your mercy for graceful operations on workers is 90 seconds Sep 26 05:28:17.344064 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: mapped 403077 bytes (393 KB) for 2 cores Sep 26 05:28:17.344064 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: *** Operational MODE: preforking *** Sep 26 05:28:17.344421 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: *** uWSGI is running in multiple interpreter mode *** Sep 26 05:28:17.344421 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: spawned uWSGI master process (pid: 81653) Sep 26 05:28:17.344663 dv-cinder-836083-14-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 26 05:28:17.349163 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: spawned uWSGI worker 1 (pid: 81663, cores: 1) Sep 26 05:28:17.349163 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: spawned uWSGI worker 2 (pid: 81664, cores: 1) Sep 26 05:28:17.349163 dv-cinder-836083-14-iscsi devstack@keystone.service[81653]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 26 05:28:21.632129 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 05:28:21.632129 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: from cryptography.utils import int_from_bytes Sep 26 05:28:21.632977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 05:28:21.632977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: from cryptography.utils import int_from_bytes Sep 26 05:28:21.632977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 05:28:21.632977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: from cryptography.utils import int_from_bytes Sep 26 05:28:21.633550 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 05:28:21.633550 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: from cryptography.utils import int_from_bytes Sep 26 05:28:21.682879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 26 05:28:21.683117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 26 05:28:21.683197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 26 05:28:21.683271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 26 05:28:21.683331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] command line args: [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 26 05:28:21.683393 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] command line args: [] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 26 05:28:21.683460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 26 05:28:21.683598 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 26 05:28:21.683740 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ================================================================================ {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 26 05:28:21.683826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ================================================================================ {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 26 05:28:21.683902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] admin_token = **** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.684050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] admin_token = **** {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.684177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] config_dir = [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.684255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] config_dir = [] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.684325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.684478 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.684609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] config_source = [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.684609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] debug = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.684767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] config_source = [] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.684843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] debug = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.684912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.685102 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.685231 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.685295 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.685358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.685421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.685591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.685718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.685939 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.686100 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.686212 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] list_limit = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.686316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] list_limit = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.686410 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] log_config_append = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.686523 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] log_config_append = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.686617 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.686709 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.686800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] log_dir = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.686903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] log_dir = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.687001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] log_file = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.687102 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] log_file = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.687202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.687319 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.687405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.687518 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.687592 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.687730 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.687805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.687919 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.687995 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.688113 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.688196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.688315 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.688395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.688461 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.688574 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.688692 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.688805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.688913 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.689018 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.689125 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.689258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.689344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.689465 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.689546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.689696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.689874 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.689956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.690079 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.690174 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.690250 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.690703 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.690836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.690836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] publish_errors = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.690965 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] publish_errors = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.691168 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.691381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.691381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.691753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.691923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.692101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.692276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.692403 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.692516 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.692596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.692666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] standard_threads = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.692772 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] standard_threads = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.692858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.692930 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.693005 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.693075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.693203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.693277 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.693409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] use_journal = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.693485 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] use_journal = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.693560 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] use_json = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.693636 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] use_json = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.693712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] use_stderr = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.693782 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] use_stderr = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.693893 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] use_syslog = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.693973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] use_syslog = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.694043 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.694117 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 05:28:21.694187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.694256 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.694335 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.694406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.694474 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.694548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.694622 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.694693 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.694858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.694980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.695058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.695128 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.695316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.695415 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.695554 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.695640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.695800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.695885 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.696046 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.696124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.696297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.696384 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.696528 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.696734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.696734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.696871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.697024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.697111 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.697266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.697347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.697516 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.697596 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.697752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.697910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.698071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.698188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.698305 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.698416 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.698609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.698723 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.698832 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.698942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.699051 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.699159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.699276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.699387 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.699495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.699604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.699722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.699843 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.699957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.700066 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.700175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.700283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.700398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.700522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.700633 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.700743 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.700858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.700968 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.701084 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.701239 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.701353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.701462 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.701570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.701678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.701787 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.702026 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.702150 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.702262 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.702420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.702542 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.702688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.702812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.702923 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.703140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.703273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.703383 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.703490 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.703607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.703718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.703826 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.703957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.704080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.704192 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.704381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.704493 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.704603 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.704718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.704828 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.704943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.705061 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.705234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.705357 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.705469 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.705578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.705686 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.705790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.705884 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.705977 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.706069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.706161 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.706259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.706360 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.706453 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.706545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.706653 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.706747 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.706844 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.706997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.707112 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.707222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.707357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.707474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.707598 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.707710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.707822 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.707933 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.708049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.708165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.708275 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.708385 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.708495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.708624 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.708758 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.708869 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.708990 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.709102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.709246 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.709360 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.709471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.709581 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.709692 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.709793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.709909 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.710021 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.710137 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.710249 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.710359 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.710469 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.710586 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.710698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.710808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.710919 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.711028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.711138 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.711255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.711366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.711477 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.711595 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.711713 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.711823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.711954 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.712066 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.712176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.712284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.712395 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.712510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.712631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.712743 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.712853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.712963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.713078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.713204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.713331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.713443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.713553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.713664 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.713773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.713883 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.714004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.714116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.714226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.714335 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.714445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.714555 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.714674 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.714785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.714908 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.715032 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.715147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.715257 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.715451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.715565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.715676 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.715792 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.715902 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.716025 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.716144 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.716263 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.716366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.716459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.716567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.716678 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.716790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.716908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.717055 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.717237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.717352 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.717475 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.717588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.717707 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.717820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.717930 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.718040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.718156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.718268 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.718386 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.718498 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.718607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.718717 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.718832 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] auth.external = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.718943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.719082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.719203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.719327 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.719450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.719563 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.719674 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] auth.external = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.719785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] auth.password = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.719897 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.720017 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] auth.token = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.720129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.720241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.720360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.720473 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.720584 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] auth.password = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.720707 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.720820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] auth.token = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.720933 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.721043 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.721170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.721284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.721408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.721522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.721633 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.721750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.721863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] credential.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.721968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.722086 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.722198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.722308 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.722420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.722531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.722643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] credential.caching = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.722765 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.722879 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.722990 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.723102 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.723221 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.723333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.723452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.723564 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.723675 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.723786 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.723904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.724017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.724495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.724611 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.724758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.724885 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.724998 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.725109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.725287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.725402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.725513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.725623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.725736 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.725847 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.725966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] federation.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.726079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.726192 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.726303 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.726413 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.726524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.726679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.726801 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] federation.caching = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.726913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.727025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.727122 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.727234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.727356 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.727470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.727617 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.727738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.727855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.727977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.728090 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.728200 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.728312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.728423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.728534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.728653 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.728779 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.728879 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.729045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.729221 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.729352 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.729556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.729675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] identity.caching = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.729774 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.729869 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.729963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.730058 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.730162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.730259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.730356 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.730450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.730545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.730640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.730741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.730928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.731092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.731199 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.731307 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.731404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.731515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.731619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.731739 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.731852 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.731950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.732099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.732218 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.732316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.732426 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.732545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.732659 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.732770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.732891 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.733005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.733286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.733406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.733521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.733634 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.733761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.733876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.733988 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.734100 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.734212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.734323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.734443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.734558 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.734670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.734781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.734893 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.735004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.735123 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.735236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.735347 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.735459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.735570 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.735683 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.735802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.735915 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.736027 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.736139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.736250 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.736361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.736481 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.736600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.736715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.736826 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.736938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.737056 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.737202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.737318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.737430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.737541 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.737653 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.737774 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.737887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.737999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.738110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.738221 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.738332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.738452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.738565 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.738676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.738788 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.738788 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.739002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.739123 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.739236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.739347 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.739458 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.739606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.739727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.739848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.739966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.740078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.740190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.740302 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.740415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.740537 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.740656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.740769 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.740879 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.740991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.741101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.741269 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.741385 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.741505 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.741618 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.741729 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.741840 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.741961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.742074 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.742185 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.742297 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.742408 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.742519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.742639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.742751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.742862 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.742973 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.743090 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.743202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.743321 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.743439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.743552 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.743662 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.743774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.743884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.744007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.744126 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.744238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.744351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.744463 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.744580 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.744700 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.744814 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.744925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.745037 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.745168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.745281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.745405 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.745519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.745630 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.745741 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.745851 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.745962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.746081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.746195 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.746307 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.746418 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.746530 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.746641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.746761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.746874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.746985 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.747097 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.747209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.747320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.747440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.747553 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.747664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.747776 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.747887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.747998 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.748117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.748221 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.748433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.748547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.748658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.748769 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.748896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.749014 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.749126 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.749292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.749403 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.749514 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] resource.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.749639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.749753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.749864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.749975 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.750086 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] resource.caching = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.750197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.750318 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.750431 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.750543 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.750654 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.750764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.750876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.750996 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.751109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.751221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.751332 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.751709 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.751826 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.752019 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.752136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] role.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.752247 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.752358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] role.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.752469 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.752580 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.752700 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] role.caching = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.752813 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.752924 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] role.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.753035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.753166 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.753292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.753406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.753518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.753629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.753741 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.753853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.753974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.754086 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.754198 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.754308 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.754419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.754531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.754651 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.754764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.754875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.754986 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.755097 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.755208 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.755328 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.755441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.755553 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.755664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.755664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.755873 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.755993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.756106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.756106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.756314 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.756426 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.756537 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.756657 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.756770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.756881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.756993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.757105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.757232 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.757357 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.757471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.757582 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.757696 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.757807 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.757935 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.758048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.758159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.758270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.758381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.758494 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.758614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.758728 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.758839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.758950 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.759064 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.759175 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.759294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.759408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.759518 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.759630 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.759742 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.759853 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.759973 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.760085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] token.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.760196 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.760308 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.760419 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.760530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.760649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] token.caching = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.760763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.760874 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.760985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.761096 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.761224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.761348 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.761463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.761574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.761685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.761796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.761908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.762027 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.762140 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.762251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.762363 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.762474 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.762585 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.762705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.762817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.762929 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.763050 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.763166 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.763278 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.763397 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.763510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.763630 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.763751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.763889 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.764003 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.764115 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.764240 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.connection = **** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.764360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.764482 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.764595 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.764706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.764817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.connection = **** {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.764928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.765041 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.765213 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.765329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.765441 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.765552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.765664 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.765781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.765902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.766017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.766128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.766239 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.766351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.766463 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.766583 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.766696 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.766809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.766921 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.767032 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.767145 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.767265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.767379 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.767491 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.767602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.767715 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.767826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.767947 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.768060 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.768173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.768285 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.768397 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.768508 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.768628 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 26 05:28:21.768741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.768854 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 05:28:21.769006 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 26 05:28:21.797961 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:21.798189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:21.798302 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:21.798431 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:21.798539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:21.798773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:21.806873 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:21.807202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:21.835767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:21.836028 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:21.836157 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:21.836383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:21.890790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:21.890978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:21.940415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:21.940641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:21.940863 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:21.941105 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.103809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.104022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.104166 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.104249 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.104399 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.104481 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.104556 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.104631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.104705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.104779 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.104868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.104945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.128440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.128613 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.128755 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.128893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.128982 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.129048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.129123 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.129236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.129326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.129405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.129534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.129617 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.129753 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.129845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.141300 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.141437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.153947 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.154180 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.154348 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.154469 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.154607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.154687 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.154838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.154962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.155116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.155200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.155264 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81664) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.155338 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.155719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.155719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81663) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 05:28:22.171936 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.172140 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 05:28:22.172386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.172512 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.172648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 05:28:22.172792 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 05:28:22.172930 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.173011 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.173128 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 05:28:22.173228 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.173406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.173532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.173670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.173752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.173918 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.174152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.183033 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.183324 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.183584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.183707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.183908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.184051 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.184189 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.184450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.190769 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.191417 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.193631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.193916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.194190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.194315 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.194453 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.194580 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.194691 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.194896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.195024 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.195160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.195241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.195455 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.195585 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.195722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.195802 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.195923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.196044 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.196233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.196361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.196481 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.196776 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.197045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.219357 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.220374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.220855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.221162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.221893 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.222183 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.226135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.226417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.226693 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.227169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.227455 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.227738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.233042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.233815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.234114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.234401 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.239108 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.239423 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.239691 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.239981 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.240246 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.240341 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.240515 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.240602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.240778 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.240864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.241043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.241155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.241382 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.241661 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.241934 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.242188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.256385 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.256800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.257621 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.257904 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.260456 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.261770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.263654 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.263992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.265164 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.265461 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.268134 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.269657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.271286 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.272774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.274474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.274768 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.275049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.275328 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.275608 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.275903 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.276030 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.276273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.276560 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.276848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.277163 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.277448 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.291441 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.291727 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.291993 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.292251 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.293164 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.293466 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.293739 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.294002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.298995 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.300497 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.300810 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.302041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.302347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.302535 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.303873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.304147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.304967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.305277 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.305567 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.305850 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.306241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.306831 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.307124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.307407 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.307702 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.308096 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.317191 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.317496 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.317777 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.318056 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.318431 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.318706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.318982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.319242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.319525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.319785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.320092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.320468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.320745 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.321021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.334248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.336274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.337101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.339161 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.339965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.341443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.342194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.342940 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.343221 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.343606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.343742 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.345066 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.345381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.345773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.350230 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.351673 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.352435 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.353923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.354554 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.354945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.355226 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.355502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.356762 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.357169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.357470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.357780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.363207 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.363498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.363773 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.365524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.365819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.366099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.370488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81664) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.370769 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.371047 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.371322 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.371591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.371879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.372159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.372421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.372686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.372896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81663) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 05:28:22.373034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.373218 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.373410 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.373507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.373693 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.373778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.373854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.374042 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.374134 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81664) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.374314 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.374600 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.374886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.375156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.375432 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.375726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.375999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.376266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.376525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81663) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 05:28:22.407555 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55dff948b9c0 pid: 81664 (default app) Sep 26 05:28:22.410380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55dff948b9c0 pid: 81663 (default app) Sep 26 05:28:22.410916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495941d8-3273-438c-9072-447c3505fe3c None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:22.411201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495941d8-3273-438c-9072-447c3505fe3c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:22.411465 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495941d8-3273-438c-9072-447c3505fe3c None None] PATH_INFO: `/v3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:22.415629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Mon Sep 26 05:28:22 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:29.441429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe21435-7d2d-4f65-8f20-8405a09c98a9 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:29.442013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe21435-7d2d-4f65-8f20-8405a09c98a9 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:29.442013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe21435-7d2d-4f65-8f20-8405a09c98a9 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:29.446293 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:28:29.452082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc280c9e-8658-4501-aeee-bb54ba9c116d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:29.452671 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc280c9e-8658-4501-aeee-bb54ba9c116d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:29.452671 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc280c9e-8658-4501-aeee-bb54ba9c116d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:29.523576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: INFO dbcounter [None req-bc280c9e-8658-4501-aeee-bb54ba9c116d None None] Registered counter for database keystone Sep 26 05:28:29.543224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG oslo_db.sqlalchemy.engines [None req-bc280c9e-8658-4501-aeee-bb54ba9c116d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81664) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 26 05:28:29.550811 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writer thread running {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 26 05:28:29.736537 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG passlib.handlers.bcrypt [None req-bc280c9e-8658-4501-aeee-bb54ba9c116d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81664) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 26 05:28:29.736904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG passlib.handlers.bcrypt [None req-bc280c9e-8658-4501-aeee-bb54ba9c116d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81664) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 26 05:28:29.797170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-bc280c9e-8658-4501-aeee-bb54ba9c116d None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:29.798726 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bc280c9e-8658-4501-aeee-bb54ba9c116d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:29.934053 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 26 05:28:30.400348 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe21435-7d2d-4f65-8f20-8405a09c98a9 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:30.470655 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: INFO dbcounter [None req-ebe21435-7d2d-4f65-8f20-8405a09c98a9 None None] Registered counter for database keystone Sep 26 05:28:30.492765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG oslo_db.sqlalchemy.engines [None req-ebe21435-7d2d-4f65-8f20-8405a09c98a9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81663) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 26 05:28:30.499579 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writer thread running {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 26 05:28:30.622325 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe21435-7d2d-4f65-8f20-8405a09c98a9 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:30.627179 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78c7ef-ee1b-4f09-ac4d-a449dcf6015e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:30.627818 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78c7ef-ee1b-4f09-ac4d-a449dcf6015e admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:30.628097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78c7ef-ee1b-4f09-ac4d-a449dcf6015e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:30.628370 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78c7ef-ee1b-4f09-ac4d-a449dcf6015e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:30.636944 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78c7ef-ee1b-4f09-ac4d-a449dcf6015e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:30.687440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5b78c7ef-ee1b-4f09-ac4d-a449dcf6015e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 26 05:28:30.687783 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78c7ef-ee1b-4f09-ac4d-a449dcf6015e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:30.697651 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Mon Sep 26 05:28:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:28:30.703928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc280c9e-8658-4501-aeee-bb54ba9c116d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:30.722441 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc280c9e-8658-4501-aeee-bb54ba9c116d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:30.728655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47ac7c3-3efd-4208-a0aa-81b4df5e19a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:30.729859 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ac7c3-3efd-4208-a0aa-81b4df5e19a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:30.730144 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ac7c3-3efd-4208-a0aa-81b4df5e19a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:30.730427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47ac7c3-3efd-4208-a0aa-81b4df5e19a5 admin admin] PATH_INFO: `/v3/projects/fb1661fcfa594d16a5b5329ef20333a6` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:30.732196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47ac7c3-3efd-4208-a0aa-81b4df5e19a5 admin admin] RBAC: Authorizing `identity:get_project(project_id=fb1661fcfa594d16a5b5329ef20333a6)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:30.784460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e47ac7c3-3efd-4208-a0aa-81b4df5e19a5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 26 05:28:30.784721 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47ac7c3-3efd-4208-a0aa-81b4df5e19a5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:30.789309 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Mon Sep 26 05:28:30 2022] GET /identity/v3/projects/fb1661fcfa594d16a5b5329ef20333a6 => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:32.974216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a75246-3e1f-4bce-9203-821a8754f619 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:32.974800 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a75246-3e1f-4bce-9203-821a8754f619 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:32.974800 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a75246-3e1f-4bce-9203-821a8754f619 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:32.978897 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:32.984747 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9add4-0891-4c1e-96d1-d71ac52d06cf None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:32.985220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9add4-0891-4c1e-96d1-d71ac52d06cf None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:32.985384 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9add4-0891-4c1e-96d1-d71ac52d06cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:33.006875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c4c9add4-0891-4c1e-96d1-d71ac52d06cf None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:33.008394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c4c9add4-0891-4c1e-96d1-d71ac52d06cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:33.017721 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 26 05:28:33.334004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a75246-3e1f-4bce-9203-821a8754f619 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:33.346884 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a75246-3e1f-4bce-9203-821a8754f619 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:33.351707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef37d4-6766-4850-8e9b-b5e119d17781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:33.352314 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef37d4-6766-4850-8e9b-b5e119d17781 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:33.352594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef37d4-6766-4850-8e9b-b5e119d17781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:33.352867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef37d4-6766-4850-8e9b-b5e119d17781 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:33.360896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef37d4-6766-4850-8e9b-b5e119d17781 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:33.362754 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef37d4-6766-4850-8e9b-b5e119d17781 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:33.372012 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Mon Sep 26 05:28:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:28:33.377735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9add4-0891-4c1e-96d1-d71ac52d06cf None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:33.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9add4-0891-4c1e-96d1-d71ac52d06cf None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:33.393740 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a847ea-ce33-4128-813c-1b2c7da5c5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:33.394365 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a847ea-ce33-4128-813c-1b2c7da5c5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:33.394639 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a847ea-ce33-4128-813c-1b2c7da5c5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:33.394886 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a847ea-ce33-4128-813c-1b2c7da5c5b8 admin admin] PATH_INFO: `/v3/users/7236c9ce10b9442581cf096a5b4cddf9` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:33.396320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a847ea-ce33-4128-813c-1b2c7da5c5b8 admin admin] RBAC: Authorizing `identity:get_user(user_id=7236c9ce10b9442581cf096a5b4cddf9)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:33.398194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a847ea-ce33-4128-813c-1b2c7da5c5b8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:33.399742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Mon Sep 26 05:28:33 2022] GET /identity/v3/users/7236c9ce10b9442581cf096a5b4cddf9 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:35.650892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1007d-5579-416f-acca-e70b2118642f None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:35.651605 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1007d-5579-416f-acca-e70b2118642f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:35.651605 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1007d-5579-416f-acca-e70b2118642f None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:35.651839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c96972-e8ed-4120-947b-d80c65d2cfea None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:35.651839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c96972-e8ed-4120-947b-d80c65d2cfea None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:35.652497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c96972-e8ed-4120-947b-d80c65d2cfea None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:35.653057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 6/10] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:28:35.653327 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 5/11] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:35.659005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3f09c-9209-43ba-9232-ed4226c950c7 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:35.659292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3f09c-9209-43ba-9232-ed4226c950c7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:35.659563 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3f09c-9209-43ba-9232-ed4226c950c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:35.659993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f681328-bdc7-4d75-a078-9a16547c5c36 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:35.660366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f681328-bdc7-4d75-a078-9a16547c5c36 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:35.660809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f681328-bdc7-4d75-a078-9a16547c5c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:35.678735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-07b3f09c-9209-43ba-9232-ed4226c950c7 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:35.679931 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-07b3f09c-9209-43ba-9232-ed4226c950c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:35.708374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG passlib.handlers.bcrypt [None req-7f681328-bdc7-4d75-a078-9a16547c5c36 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81663) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 26 05:28:35.708767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG passlib.handlers.bcrypt [None req-7f681328-bdc7-4d75-a078-9a16547c5c36 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81663) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 26 05:28:35.757703 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-7f681328-bdc7-4d75-a078-9a16547c5c36 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:35.759432 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7f681328-bdc7-4d75-a078-9a16547c5c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:35.775118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 7/12] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:28:35.867439 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 6/13] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:28:36.042143 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3f09c-9209-43ba-9232-ed4226c950c7 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:36.068079 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:36.129345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f681328-bdc7-4d75-a078-9a16547c5c36 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:36.155973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3f09c-9209-43ba-9232-ed4226c950c7 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:36.189662 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7375c56-12cd-4cef-87a6-af34d988a77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:36.190308 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7375c56-12cd-4cef-87a6-af34d988a77d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:36.190585 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7375c56-12cd-4cef-87a6-af34d988a77d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:36.190866 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7375c56-12cd-4cef-87a6-af34d988a77d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:36.202399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7375c56-12cd-4cef-87a6-af34d988a77d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:28:36.211370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b7375c56-12cd-4cef-87a6-af34d988a77d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:28:36.211370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:28:36.211370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:28:36.211370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:28:36.211370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:28:36.211370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:28:36.211370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:28:36.211370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:28:36.211370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:28:36.211370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:28:36.211370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:28:36.211370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:28:36.211370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:28:36.211370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:28:36.211370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:28:36.211370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:28:36.212312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:28:36.212312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:28:36.212312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:28:36.212312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:28:36.212312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:28:36.212312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:28:36.212312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:28:36.212312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:28:36.212312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:28:36.212312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:28:36.212312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:28:36.212312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:28:36.212312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:28:36.212312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:28:36.212312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:28:36.212312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:28:36.213165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:28:36.213165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:28:36.213165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:28:36.213165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:28:36.213165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:28:36.213165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:28:36.213165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:28:36.213165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:28:36.213165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:28:36.213165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:28:36.213165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:28:36.213165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:28:36.213165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 8/14] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:28:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:28:36.218670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7375c56-12cd-4cef-87a6-af34d988a77d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:36.234655 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f681328-bdc7-4d75-a078-9a16547c5c36 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:36.237179 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:36.273302 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44cbaf-0ab3-46c9-aa2b-7f97c539f43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:36.274198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44cbaf-0ab3-46c9-aa2b-7f97c539f43a None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:36.274470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44cbaf-0ab3-46c9-aa2b-7f97c539f43a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:36.274737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44cbaf-0ab3-46c9-aa2b-7f97c539f43a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:36.321826 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:36.340918 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7375c56-12cd-4cef-87a6-af34d988a77d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:36.351835 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44cbaf-0ab3-46c9-aa2b-7f97c539f43a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:36.353470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44cbaf-0ab3-46c9-aa2b-7f97c539f43a None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:36.379238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8e00e-9f52-40cf-acb9-90c921280fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:36.380090 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8e00e-9f52-40cf-acb9-90c921280fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:36.380447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8e00e-9f52-40cf-acb9-90c921280fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:36.380905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8e00e-9f52-40cf-acb9-90c921280fe4 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:36.381745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8e00e-9f52-40cf-acb9-90c921280fe4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:36.382167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8e00e-9f52-40cf-acb9-90c921280fe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:36.383731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8e00e-9f52-40cf-acb9-90c921280fe4 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:36.397745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 9/15] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:28:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:36.404000 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8e00e-9f52-40cf-acb9-90c921280fe4 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:36.460417 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 7/16] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:28:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:28:36.466253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44cbaf-0ab3-46c9-aa2b-7f97c539f43a None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:36.468959 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:36.503933 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8e00e-9f52-40cf-acb9-90c921280fe4 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:36.538185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6206944-1a19-41f9-a341-ebc2d79cbadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:36.538817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6206944-1a19-41f9-a341-ebc2d79cbadf None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:36.539095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6206944-1a19-41f9-a341-ebc2d79cbadf None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:36.539364 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6206944-1a19-41f9-a341-ebc2d79cbadf None admin] PATH_INFO: `/v3/users/7236c9ce10b9442581cf096a5b4cddf9` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:36.540513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6206944-1a19-41f9-a341-ebc2d79cbadf None admin] RBAC: Authorizing `identity:get_user(user_id=7236c9ce10b9442581cf096a5b4cddf9)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:36.542005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6206944-1a19-41f9-a341-ebc2d79cbadf None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:36.543576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 10/17] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:28:36 2022] GET /identity/v3/users/7236c9ce10b9442581cf096a5b4cddf9 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:36.548985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6206944-1a19-41f9-a341-ebc2d79cbadf None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:36.563486 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:36.566492 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44cbaf-0ab3-46c9-aa2b-7f97c539f43a None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:36.603787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9c3ce-fe3f-45e4-97eb-06e8d873060a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:36.604562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9c3ce-fe3f-45e4-97eb-06e8d873060a None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:36.604844 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9c3ce-fe3f-45e4-97eb-06e8d873060a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:36.605109 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9c3ce-fe3f-45e4-97eb-06e8d873060a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:36.623808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9c3ce-fe3f-45e4-97eb-06e8d873060a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:36.625354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9c3ce-fe3f-45e4-97eb-06e8d873060a None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:36.644135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-8ab9c3ce-fe3f-45e4-97eb-06e8d873060a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:28:36.644135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:28:36.644135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:28:36.644135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:28:36.644135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:28:36.644135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:28:36.644135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:28:36.644135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:28:36.644135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:28:36.644135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:28:36.644135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:28:36.644135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:28:36.644135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:28:36.644135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:28:36.644135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:28:36.644135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:28:36.645045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:28:36.645045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:28:36.645045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:28:36.645045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:28:36.645045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:28:36.645045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:28:36.645045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:28:36.645045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:28:36.645045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:28:36.645045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:28:36.645045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:28:36.645045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:28:36.645045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:28:36.645045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:28:36.645045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:28:36.645045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:28:36.645045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:28:36.645889 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:28:36.645889 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:28:36.645889 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:28:36.645889 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 8/18] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:28:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:28:36.650093 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6206944-1a19-41f9-a341-ebc2d79cbadf None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:36.651370 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9c3ce-fe3f-45e4-97eb-06e8d873060a None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:36.690364 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d5f3a-955f-4e62-bdb1-b194d84035e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:36.691018 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d5f3a-955f-4e62-bdb1-b194d84035e5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:36.691290 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d5f3a-955f-4e62-bdb1-b194d84035e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:36.691560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d5f3a-955f-4e62-bdb1-b194d84035e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:36.692815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d5f3a-955f-4e62-bdb1-b194d84035e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:36.694349 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d5f3a-955f-4e62-bdb1-b194d84035e5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:36.695965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 11/19] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:28:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:28:36.702169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d5f3a-955f-4e62-bdb1-b194d84035e5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:36.712617 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:36.760425 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9c3ce-fe3f-45e4-97eb-06e8d873060a None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:36.794415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:36.803524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b10cc-9b8a-4b78-bd10-08640559e123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:36.804170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b10cc-9b8a-4b78-bd10-08640559e123 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:36.804443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b10cc-9b8a-4b78-bd10-08640559e123 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:36.804710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b10cc-9b8a-4b78-bd10-08640559e123 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:36.805423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b10cc-9b8a-4b78-bd10-08640559e123 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:36.805732 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b10cc-9b8a-4b78-bd10-08640559e123 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:36.807528 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b10cc-9b8a-4b78-bd10-08640559e123 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:36.820208 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d5f3a-955f-4e62-bdb1-b194d84035e5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:36.827714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 9/20] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:28:36.859062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7ed6b-1c08-4f43-8294-c777ed9b8642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:36.859598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7ed6b-1c08-4f43-8294-c777ed9b8642 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:36.859873 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7ed6b-1c08-4f43-8294-c777ed9b8642 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:36.860138 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7ed6b-1c08-4f43-8294-c777ed9b8642 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:36.860883 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7ed6b-1c08-4f43-8294-c777ed9b8642 None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.domain.id=default, user.id=7236c9ce10b9442581cf096a5b4cddf9) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:36.861216 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7ed6b-1c08-4f43-8294-c777ed9b8642 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:36.862712 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7ed6b-1c08-4f43-8294-c777ed9b8642 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:36.870898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 12/21] 10.222.0.57 () {56 vars in 1322 bytes} [Mon Sep 26 05:28:36 2022] GET /identity/v3/role_assignments?user.id=7236c9ce10b9442581cf096a5b4cddf9&scope.domain.id=default&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 233 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:28:39.474647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895a2a3-4c66-42b3-a04b-ad11fd584958 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:39.475167 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895a2a3-4c66-42b3-a04b-ad11fd584958 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:39.475167 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895a2a3-4c66-42b3-a04b-ad11fd584958 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:39.476530 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 10/22] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:39.482184 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0246ff8-a38b-4959-8c17-203ea6d6d1e1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:39.482594 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0246ff8-a38b-4959-8c17-203ea6d6d1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:39.482759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0246ff8-a38b-4959-8c17-203ea6d6d1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:39.510247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c0246ff8-a38b-4959-8c17-203ea6d6d1e1 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:39.511391 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c0246ff8-a38b-4959-8c17-203ea6d6d1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:39.580278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518453c5-f350-4119-94be-6c2c2f464c34 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:39.580933 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518453c5-f350-4119-94be-6c2c2f464c34 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:39.581429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518453c5-f350-4119-94be-6c2c2f464c34 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:39.583275 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 11/23] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:39.587196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 13/24] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:28:39.589229 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17566cbc-2cf8-4c5e-b6dc-c60d3e03e14c None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:39.589751 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17566cbc-2cf8-4c5e-b6dc-c60d3e03e14c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:39.590222 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17566cbc-2cf8-4c5e-b6dc-c60d3e03e14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:39.634290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-17566cbc-2cf8-4c5e-b6dc-c60d3e03e14c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:39.636143 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-17566cbc-2cf8-4c5e-b6dc-c60d3e03e14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:39.731588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 12/25] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:28:39.742520 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19419cd-299e-4224-a8ff-70736ed016b0 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:39.742802 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19419cd-299e-4224-a8ff-70736ed016b0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:39.743245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19419cd-299e-4224-a8ff-70736ed016b0 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:39.744773 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 14/26] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:39.750626 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60bbfa-1aab-4614-a0f9-7767fb0625f3 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:39.750919 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60bbfa-1aab-4614-a0f9-7767fb0625f3 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:39.751194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60bbfa-1aab-4614-a0f9-7767fb0625f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:39.778365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-7e60bbfa-1aab-4614-a0f9-7767fb0625f3 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:39.779569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7e60bbfa-1aab-4614-a0f9-7767fb0625f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:39.835161 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:39.867151 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 13/27] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:28:40.075754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19419cd-299e-4224-a8ff-70736ed016b0 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:40.167965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:40.201500 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a195a1-2884-4314-8f15-618956fdf3be None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:40.202698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a195a1-2884-4314-8f15-618956fdf3be None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:40.203509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19419cd-299e-4224-a8ff-70736ed016b0 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:40.205693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a195a1-2884-4314-8f15-618956fdf3be None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:40.207284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 14/28] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:40.219507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a195a1-2884-4314-8f15-618956fdf3be None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:40.261576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1fea2-394a-4e43-a305-4093c65b301b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:40.262227 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1fea2-394a-4e43-a305-4093c65b301b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:40.262501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1fea2-394a-4e43-a305-4093c65b301b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:40.262770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1fea2-394a-4e43-a305-4093c65b301b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:40.281036 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1fea2-394a-4e43-a305-4093c65b301b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:40.282561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1fea2-394a-4e43-a305-4093c65b301b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:40.311556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-afd1fea2-394a-4e43-a305-4093c65b301b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:28:40.311556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:28:40.311556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:28:40.311556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:28:40.311556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:28:40.311556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:28:40.311556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:28:40.311556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:28:40.311556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:28:40.311556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:28:40.311556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:28:40.311556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:28:40.311556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:28:40.311556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:28:40.311556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:28:40.311556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:28:40.312477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:28:40.312477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:28:40.312477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:28:40.312477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:28:40.312477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:28:40.312477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:28:40.312477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:28:40.312477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:28:40.312477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:28:40.312477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:28:40.312477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:28:40.312477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:28:40.312477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:28:40.312477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:28:40.312477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:28:40.312477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:28:40.312477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:28:40.313326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:28:40.313326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:28:40.313326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:28:40.314068 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 15/29] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:28:40.315263 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a6eee-879f-44d6-97a2-6f86f1413515 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:40.316827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a6eee-879f-44d6-97a2-6f86f1413515 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:40.317209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a6eee-879f-44d6-97a2-6f86f1413515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:40.346577 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-2d1a6eee-879f-44d6-97a2-6f86f1413515 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:40.348246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2d1a6eee-879f-44d6-97a2-6f86f1413515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:40.380189 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a195a1-2884-4314-8f15-618956fdf3be None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:40.422149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b26c64-76a0-47c7-8fd7-392a18269945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:40.422818 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b26c64-76a0-47c7-8fd7-392a18269945 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:40.423075 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b26c64-76a0-47c7-8fd7-392a18269945 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:40.423347 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b26c64-76a0-47c7-8fd7-392a18269945 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:40.425320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 16/30] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:28:40.426593 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a6eee-879f-44d6-97a2-6f86f1413515 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:40.448956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b26c64-76a0-47c7-8fd7-392a18269945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:28:40.459002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-43b26c64-76a0-47c7-8fd7-392a18269945 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:28:40.459002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:28:40.459002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:28:40.459002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:28:40.459002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:28:40.459002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:28:40.459002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:28:40.459002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:28:40.459002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:28:40.459002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:28:40.459002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:28:40.459002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:28:40.459002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:28:40.459002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:28:40.459002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:28:40.459002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:28:40.459864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:28:40.459864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:28:40.459864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:28:40.459864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:28:40.459864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:28:40.459864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:28:40.459864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:28:40.459864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:28:40.459864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:28:40.459864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:28:40.459864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:28:40.459864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:28:40.459864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:28:40.459864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:28:40.459864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:28:40.459864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:28:40.460691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:28:40.460691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:28:40.460691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:28:40.460691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:28:40.460691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:28:40.460691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:28:40.460691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:28:40.460691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:28:40.460691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:28:40.460691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:28:40.460691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:28:40.460691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:28:40.465848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 15/31] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:28:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:28:40.467832 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b26c64-76a0-47c7-8fd7-392a18269945 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:40.527962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:40.539563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:40.615276 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a6eee-879f-44d6-97a2-6f86f1413515 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:40.660903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b26c64-76a0-47c7-8fd7-392a18269945 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:40.678501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e06fe-0aef-461b-ad87-1e7acffda9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:40.679706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e06fe-0aef-461b-ad87-1e7acffda9ff None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:40.680411 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e06fe-0aef-461b-ad87-1e7acffda9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:40.682157 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e06fe-0aef-461b-ad87-1e7acffda9ff None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:40.683040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e06fe-0aef-461b-ad87-1e7acffda9ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:40.683736 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e06fe-0aef-461b-ad87-1e7acffda9ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:40.686311 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e06fe-0aef-461b-ad87-1e7acffda9ff None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:40.724889 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 17/32] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:40.730758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e06fe-0aef-461b-ad87-1e7acffda9ff None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:40.734552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bc55f-f675-43ad-a646-1858a9bc22c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:40.736536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bc55f-f675-43ad-a646-1858a9bc22c7 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:40.738249 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bc55f-f675-43ad-a646-1858a9bc22c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:40.738752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bc55f-f675-43ad-a646-1858a9bc22c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:40.740067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5bc55f-f675-43ad-a646-1858a9bc22c7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:40.742849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5bc55f-f675-43ad-a646-1858a9bc22c7 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:40.868766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e06fe-0aef-461b-ad87-1e7acffda9ff None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:40.886241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:40.901187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 16/33] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Sep 26 05:28:40 2022] POST /identity/v3/roles => generated 225 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:28:40.901187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bc55f-f675-43ad-a646-1858a9bc22c7 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:40.915124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00890cee-ca5d-488c-be42-f4fc3eb51b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:40.916168 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00890cee-ca5d-488c-be42-f4fc3eb51b68 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:40.916588 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00890cee-ca5d-488c-be42-f4fc3eb51b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:40.917000 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00890cee-ca5d-488c-be42-f4fc3eb51b68 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:40.917902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00890cee-ca5d-488c-be42-f4fc3eb51b68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:40.918397 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00890cee-ca5d-488c-be42-f4fc3eb51b68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:40.920276 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00890cee-ca5d-488c-be42-f4fc3eb51b68 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:40.930597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 18/34] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:28:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:40.936898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00890cee-ca5d-488c-be42-f4fc3eb51b68 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:41.010115 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:41.016288 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bc55f-f675-43ad-a646-1858a9bc22c7 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:41.059174 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa00727-07d1-4607-b76b-e1b55ef9fe44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:41.059849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa00727-07d1-4607-b76b-e1b55ef9fe44 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:41.060187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa00727-07d1-4607-b76b-e1b55ef9fe44 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:41.060473 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa00727-07d1-4607-b76b-e1b55ef9fe44 None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:41.061555 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa00727-07d1-4607-b76b-e1b55ef9fe44 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:41.063238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa00727-07d1-4607-b76b-e1b55ef9fe44 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:41.067302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00890cee-ca5d-488c-be42-f4fc3eb51b68 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:41.103242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 17/35] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 26 05:28:40 2022] POST /identity/v3/projects => generated 301 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:28:41.103242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b7d8c-b827-4a77-bc3a-ec0798e9c95e None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:41.103242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b7d8c-b827-4a77-bc3a-ec0798e9c95e None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:41.103242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b7d8c-b827-4a77-bc3a-ec0798e9c95e None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:41.103242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 18/36] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:41.103242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b7d8c-b827-4a77-bc3a-ec0798e9c95e None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:41.148734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16640b2-fb74-4b3a-b511-18ee066c43c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:41.150136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16640b2-fb74-4b3a-b511-18ee066c43c5 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:41.150670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16640b2-fb74-4b3a-b511-18ee066c43c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:41.151167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16640b2-fb74-4b3a-b511-18ee066c43c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:41.152150 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16640b2-fb74-4b3a-b511-18ee066c43c5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:41.154107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16640b2-fb74-4b3a-b511-18ee066c43c5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:41.245993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b7d8c-b827-4a77-bc3a-ec0798e9c95e None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:41.285192 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 19/37] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Sep 26 05:28:40 2022] POST /identity/v3/roles => generated 219 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:28:41.285809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8e260-3710-4f4c-963d-f6b79acb8519 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:41.286099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8e260-3710-4f4c-963d-f6b79acb8519 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:41.286380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8e260-3710-4f4c-963d-f6b79acb8519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:41.291244 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a13ec-42b9-4501-9185-b30d70cedac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:41.292291 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a13ec-42b9-4501-9185-b30d70cedac7 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:41.292750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a13ec-42b9-4501-9185-b30d70cedac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:41.293250 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a13ec-42b9-4501-9185-b30d70cedac7 None admin] PATH_INFO: `/v3/users/7236c9ce10b9442581cf096a5b4cddf9` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:41.295270 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a13ec-42b9-4501-9185-b30d70cedac7 None admin] RBAC: Authorizing `identity:get_user(user_id=7236c9ce10b9442581cf096a5b4cddf9)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:41.297166 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a13ec-42b9-4501-9185-b30d70cedac7 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:41.299479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 19/38] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:28:41 2022] GET /identity/v3/users/7236c9ce10b9442581cf096a5b4cddf9 => generated 247 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:41.301170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06903a70-d35c-4f9b-9732-04d526d80155 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:41.301752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06903a70-d35c-4f9b-9732-04d526d80155 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:41.302165 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06903a70-d35c-4f9b-9732-04d526d80155 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:41.308694 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 20/39] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:41.310254 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06903a70-d35c-4f9b-9732-04d526d80155 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:41.314755 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a2d8e260-3710-4f4c-963d-f6b79acb8519 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:41.316606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a2d8e260-3710-4f4c-963d-f6b79acb8519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:41.335666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:41.360959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:41.412799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 20/40] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:28:41.413906 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa97027-6daa-4fdf-96bf-4a63cb9faabd None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:41.414202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa97027-6daa-4fdf-96bf-4a63cb9faabd None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:41.414483 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa97027-6daa-4fdf-96bf-4a63cb9faabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:41.450122 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-baa97027-6daa-4fdf-96bf-4a63cb9faabd None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:41.451823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-baa97027-6daa-4fdf-96bf-4a63cb9faabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:41.459144 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06903a70-d35c-4f9b-9732-04d526d80155 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:41.507375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf056c0-9b9c-45e6-b6da-918a5f22f026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:41.508028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf056c0-9b9c-45e6-b6da-918a5f22f026 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:41.508309 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf056c0-9b9c-45e6-b6da-918a5f22f026 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:41.508582 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf056c0-9b9c-45e6-b6da-918a5f22f026 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:41.510043 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf056c0-9b9c-45e6-b6da-918a5f22f026 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:41.512356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf056c0-9b9c-45e6-b6da-918a5f22f026 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:41.516188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 21/41] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:28:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:41.520703 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf056c0-9b9c-45e6-b6da-918a5f22f026 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:41.652996 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:41.659010 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 21/42] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:28:41.671422 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf056c0-9b9c-45e6-b6da-918a5f22f026 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:41.707338 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91471d-defa-4c97-80ac-78b0939ef595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:41.708002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91471d-defa-4c97-80ac-78b0939ef595 None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:41.708278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91471d-defa-4c97-80ac-78b0939ef595 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:41.708550 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91471d-defa-4c97-80ac-78b0939ef595 None admin] PATH_INFO: `/v3/domains/default/users/7236c9ce10b9442581cf096a5b4cddf9/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:41.710526 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91471d-defa-4c97-80ac-78b0939ef595 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7236c9ce10b9442581cf096a5b4cddf9, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:41.711961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91471d-defa-4c97-80ac-78b0939ef595 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:41.732840 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 22/43] 10.222.0.57 () {58 vars in 1361 bytes} [Mon Sep 26 05:28:41 2022] PUT /identity/v3/domains/default/users/7236c9ce10b9442581cf096a5b4cddf9/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:28:41.739383 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa97027-6daa-4fdf-96bf-4a63cb9faabd None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:41.861440 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:41.867282 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa97027-6daa-4fdf-96bf-4a63cb9faabd None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:41.915059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66193cab-1cbc-4189-96a6-8b120bf7a2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:41.916069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66193cab-1cbc-4189-96a6-8b120bf7a2b2 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:41.916658 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66193cab-1cbc-4189-96a6-8b120bf7a2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:41.917127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66193cab-1cbc-4189-96a6-8b120bf7a2b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:41.918241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66193cab-1cbc-4189-96a6-8b120bf7a2b2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:41.920339 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66193cab-1cbc-4189-96a6-8b120bf7a2b2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:41.930501 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91471d-defa-4c97-80ac-78b0939ef595 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:41.940197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 22/44] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Sep 26 05:28:41 2022] POST /identity/v3/roles => generated 223 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:28:42.063355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91471d-defa-4c97-80ac-78b0939ef595 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:42.104520 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d1450-3e01-4125-865c-2527f99e0fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:42.105441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d1450-3e01-4125-865c-2527f99e0fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:42.105837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d1450-3e01-4125-865c-2527f99e0fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:42.106172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d1450-3e01-4125-865c-2527f99e0fc8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:42.107667 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d1450-3e01-4125-865c-2527f99e0fc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:42.109206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d1450-3e01-4125-865c-2527f99e0fc8 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:42.110858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 23/45] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:28:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:42.117705 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66193cab-1cbc-4189-96a6-8b120bf7a2b2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:42.252666 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66193cab-1cbc-4189-96a6-8b120bf7a2b2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:42.288127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365136c3-76b0-422f-ac61-4ec8bc80ff44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:42.288749 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365136c3-76b0-422f-ac61-4ec8bc80ff44 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:42.289035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365136c3-76b0-422f-ac61-4ec8bc80ff44 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:42.289352 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365136c3-76b0-422f-ac61-4ec8bc80ff44 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:42.290028 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365136c3-76b0-422f-ac61-4ec8bc80ff44 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:42.291508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365136c3-76b0-422f-ac61-4ec8bc80ff44 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:42.325504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 23/46] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 26 05:28:42 2022] POST /identity/v3/projects => generated 312 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:28:43.944174 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec0bf4-d30b-44f4-96c5-3123eb60d63e None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:43.944773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec0bf4-d30b-44f4-96c5-3123eb60d63e None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:43.944773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec0bf4-d30b-44f4-96c5-3123eb60d63e None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:43.946043 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 24/47] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:43.951869 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6f57b-2149-47d4-9ed6-ffe82cb372b2 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:43.952306 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6f57b-2149-47d4-9ed6-ffe82cb372b2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:43.952557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6f57b-2149-47d4-9ed6-ffe82cb372b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:43.971630 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-14f6f57b-2149-47d4-9ed6-ffe82cb372b2 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:43.972823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-14f6f57b-2149-47d4-9ed6-ffe82cb372b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:43.976209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:44.045762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 24/48] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:28:44.323631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec0bf4-d30b-44f4-96c5-3123eb60d63e None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:44.339900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:44.437156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec0bf4-d30b-44f4-96c5-3123eb60d63e None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:44.472999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca58d4-0e7c-4e4f-8cae-42eb6efab465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:44.474031 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca58d4-0e7c-4e4f-8cae-42eb6efab465 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:44.474307 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca58d4-0e7c-4e4f-8cae-42eb6efab465 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:44.474581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca58d4-0e7c-4e4f-8cae-42eb6efab465 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:44.479239 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daba3b-74cf-4feb-b653-82bdc90697da None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:44.479501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daba3b-74cf-4feb-b653-82bdc90697da None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:44.479780 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daba3b-74cf-4feb-b653-82bdc90697da None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:44.481172 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 25/49] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:44.486256 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca58d4-0e7c-4e4f-8cae-42eb6efab465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:28:44.486438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7b46f-49e9-4c7b-89a2-5d3296055c25 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:44.486731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7b46f-49e9-4c7b-89a2-5d3296055c25 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:44.487010 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7b46f-49e9-4c7b-89a2-5d3296055c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:44.491020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-f9ca58d4-0e7c-4e4f-8cae-42eb6efab465 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:28:44.491020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:28:44.491020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:28:44.491020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:28:44.491020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:28:44.491020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:28:44.491020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:28:44.491020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:28:44.491020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:28:44.491020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:28:44.491020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:28:44.491020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:28:44.491020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:28:44.491020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:28:44.491020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:28:44.491020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:28:44.491850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:28:44.491850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:28:44.491850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:28:44.491850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:28:44.491850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:28:44.491850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:28:44.491850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:28:44.491850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:28:44.491850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:28:44.491850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:28:44.491850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:28:44.491850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:28:44.491850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:28:44.491850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:28:44.491850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:28:44.491850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:28:44.492609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:28:44.492609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:28:44.492609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:28:44.492609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:28:44.492609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:28:44.492609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:28:44.492609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:28:44.492609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:28:44.492609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:28:44.492609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:28:44.492609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:28:44.492609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:28:44.492609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 25/50] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:28:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:28:44.498067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca58d4-0e7c-4e4f-8cae-42eb6efab465 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:44.506479 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-40d7b46f-49e9-4c7b-89a2-5d3296055c25 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:44.507705 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-40d7b46f-49e9-4c7b-89a2-5d3296055c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:44.567210 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:44.577023 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 26/51] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:28:44.597398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca58d4-0e7c-4e4f-8cae-42eb6efab465 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:44.632552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a37d3f-c103-482f-8917-b9cbad1f1d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:44.633188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a37d3f-c103-482f-8917-b9cbad1f1d0c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:44.633470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a37d3f-c103-482f-8917-b9cbad1f1d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:44.633741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a37d3f-c103-482f-8917-b9cbad1f1d0c None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:44.634406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a37d3f-c103-482f-8917-b9cbad1f1d0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:44.634731 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a37d3f-c103-482f-8917-b9cbad1f1d0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:44.636106 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a37d3f-c103-482f-8917-b9cbad1f1d0c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:44.648255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 26/52] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:28:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:28:44.653989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7b46f-49e9-4c7b-89a2-5d3296055c25 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:44.727590 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:44.766171 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7b46f-49e9-4c7b-89a2-5d3296055c25 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:44.802446 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb1a33-3261-4931-a6cc-69bc0a7f87cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:44.803081 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb1a33-3261-4931-a6cc-69bc0a7f87cd None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:44.803358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb1a33-3261-4931-a6cc-69bc0a7f87cd None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:44.803622 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb1a33-3261-4931-a6cc-69bc0a7f87cd None admin] PATH_INFO: `/v3/users/7236c9ce10b9442581cf096a5b4cddf9` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:44.804776 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb1a33-3261-4931-a6cc-69bc0a7f87cd None admin] RBAC: Authorizing `identity:get_user(user_id=7236c9ce10b9442581cf096a5b4cddf9)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:44.806254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb1a33-3261-4931-a6cc-69bc0a7f87cd None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:44.807821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 27/53] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:28:44 2022] GET /identity/v3/users/7236c9ce10b9442581cf096a5b4cddf9 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:44.813812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a37d3f-c103-482f-8917-b9cbad1f1d0c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:44.851651 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb1a33-3261-4931-a6cc-69bc0a7f87cd None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:44.916450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a37d3f-c103-482f-8917-b9cbad1f1d0c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:44.957884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb1a33-3261-4931-a6cc-69bc0a7f87cd None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:44.996959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067e089-f251-4ccf-b6d9-5d14ad9faada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:44.997737 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067e089-f251-4ccf-b6d9-5d14ad9faada None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:44.998019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067e089-f251-4ccf-b6d9-5d14ad9faada None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:44.998293 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067e089-f251-4ccf-b6d9-5d14ad9faada None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:44.999511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067e089-f251-4ccf-b6d9-5d14ad9faada None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:45.001029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067e089-f251-4ccf-b6d9-5d14ad9faada None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:45.002713 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 28/54] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:28:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:28:45.008860 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067e089-f251-4ccf-b6d9-5d14ad9faada None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:45.018420 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:45.058627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b52f7-f66f-4fcf-9931-838582acb966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:45.059335 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b52f7-f66f-4fcf-9931-838582acb966 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:45.059542 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b52f7-f66f-4fcf-9931-838582acb966 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:45.059812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b52f7-f66f-4fcf-9931-838582acb966 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:45.061070 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b52f7-f66f-4fcf-9931-838582acb966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:45.062555 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b52f7-f66f-4fcf-9931-838582acb966 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:45.064117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 27/55] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:28:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:45.070047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b52f7-f66f-4fcf-9931-838582acb966 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:45.072856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:45.112024 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067e089-f251-4ccf-b6d9-5d14ad9faada None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:45.146722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92290563-28df-454f-a869-6ac5e6b54bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:45.147655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92290563-28df-454f-a869-6ac5e6b54bb6 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:45.147998 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92290563-28df-454f-a869-6ac5e6b54bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:45.148278 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92290563-28df-454f-a869-6ac5e6b54bb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:45.148957 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92290563-28df-454f-a869-6ac5e6b54bb6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:45.150475 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92290563-28df-454f-a869-6ac5e6b54bb6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:45.168831 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b52f7-f66f-4fcf-9931-838582acb966 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:45.177583 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 29/56] 10.222.0.57 () {60 vars in 1140 bytes} [Mon Sep 26 05:28:45 2022] POST /identity/v3/projects => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:28:45.205922 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8142a7b-6d96-451b-ba34-4f57a6fa326f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:45.206561 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8142a7b-6d96-451b-ba34-4f57a6fa326f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:45.206813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8142a7b-6d96-451b-ba34-4f57a6fa326f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:45.207114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8142a7b-6d96-451b-ba34-4f57a6fa326f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:45.207868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8142a7b-6d96-451b-ba34-4f57a6fa326f None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.domain.id=default, user.id=7236c9ce10b9442581cf096a5b4cddf9) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:45.208154 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8142a7b-6d96-451b-ba34-4f57a6fa326f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:45.209683 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8142a7b-6d96-451b-ba34-4f57a6fa326f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:45.221069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 28/57] 10.222.0.57 () {56 vars in 1322 bytes} [Mon Sep 26 05:28:45 2022] GET /identity/v3/role_assignments?user.id=7236c9ce10b9442581cf096a5b4cddf9&scope.domain.id=default&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:28:47.338871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556599c9-bdc8-44f5-9f98-fc7aa64fab68 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:47.339415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556599c9-bdc8-44f5-9f98-fc7aa64fab68 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:47.339415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556599c9-bdc8-44f5-9f98-fc7aa64fab68 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:47.340789 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 30/58] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:28:47.346354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984ce5a-168b-4f06-a2da-417e026cff69 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:47.346791 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984ce5a-168b-4f06-a2da-417e026cff69 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:47.346930 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984ce5a-168b-4f06-a2da-417e026cff69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:47.366103 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-2984ce5a-168b-4f06-a2da-417e026cff69 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:47.367642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2984ce5a-168b-4f06-a2da-417e026cff69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:47.428371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:47.436139 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 29/59] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:28:47.700445 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556599c9-bdc8-44f5-9f98-fc7aa64fab68 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:47.765720 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:47.811120 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556599c9-bdc8-44f5-9f98-fc7aa64fab68 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:47.846893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d469d-d7d9-4e9b-98cd-b8718133ea87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:47.847536 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d469d-d7d9-4e9b-98cd-b8718133ea87 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:47.847823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d469d-d7d9-4e9b-98cd-b8718133ea87 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:47.848099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d469d-d7d9-4e9b-98cd-b8718133ea87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:47.849667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d469d-d7d9-4e9b-98cd-b8718133ea87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:47.851150 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d469d-d7d9-4e9b-98cd-b8718133ea87 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:47.852744 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 31/60] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:28:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:47.858927 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984ce5a-168b-4f06-a2da-417e026cff69 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:47.963961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984ce5a-168b-4f06-a2da-417e026cff69 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:47.999347 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe617c5-244e-4fc7-9373-bb34e9ca2479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:47.999955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe617c5-244e-4fc7-9373-bb34e9ca2479 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:48.000241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe617c5-244e-4fc7-9373-bb34e9ca2479 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:48.000516 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe617c5-244e-4fc7-9373-bb34e9ca2479 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:48.001190 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe617c5-244e-4fc7-9373-bb34e9ca2479 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:48.002658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe617c5-244e-4fc7-9373-bb34e9ca2479 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:48.029654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 30/61] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:28:47 2022] POST /identity/v3/users => generated 275 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:28:51.409561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeaeace-d356-4b2e-b9a1-aa8d23b71f62 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:51.410685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeaeace-d356-4b2e-b9a1-aa8d23b71f62 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:51.411178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeaeace-d356-4b2e-b9a1-aa8d23b71f62 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:51.412833 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 32/62] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:51.418997 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bdc8e-7567-4387-8a52-c18b28d0d92b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:51.420013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bdc8e-7567-4387-8a52-c18b28d0d92b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:51.420504 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bdc8e-7567-4387-8a52-c18b28d0d92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:51.450179 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-460bdc8e-7567-4387-8a52-c18b28d0d92b None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:51.451871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-460bdc8e-7567-4387-8a52-c18b28d0d92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:51.497394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95e746-0aa6-4a10-aa60-38fd19be39aa None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:51.497999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95e746-0aa6-4a10-aa60-38fd19be39aa None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:51.498510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95e746-0aa6-4a10-aa60-38fd19be39aa None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:51.500169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 33/63] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:28:51.506284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe3022-e829-4aa7-93cb-fde1639d3fe0 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:51.506905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe3022-e829-4aa7-93cb-fde1639d3fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:51.507371 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe3022-e829-4aa7-93cb-fde1639d3fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:51.517072 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:51.527159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-2cfe3022-e829-4aa7-93cb-fde1639d3fe0 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:51.528710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2cfe3022-e829-4aa7-93cb-fde1639d3fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:51.535397 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 31/64] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:28:51.537051 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439081f2-a79e-40cb-a29d-2b0160fbee7d None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:51.537597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439081f2-a79e-40cb-a29d-2b0160fbee7d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:51.538089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439081f2-a79e-40cb-a29d-2b0160fbee7d None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:51.539539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 32/65] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:51.548728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7387f-6985-4e21-8c42-5095792f302c None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:51.549336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7387f-6985-4e21-8c42-5095792f302c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:51.551202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7387f-6985-4e21-8c42-5095792f302c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:51.582663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-3cd7387f-6985-4e21-8c42-5095792f302c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:51.585537 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-3cd7387f-6985-4e21-8c42-5095792f302c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:51.663007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 34/66] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:28:51.689661 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 33/67] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:28:51.856928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe3022-e829-4aa7-93cb-fde1639d3fe0 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:51.914932 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:51.970395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe3022-e829-4aa7-93cb-fde1639d3fe0 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:51.974515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7387f-6985-4e21-8c42-5095792f302c None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:52.013893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09c7ac-9550-4f92-8302-8c944f848a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:52.014565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09c7ac-9550-4f92-8302-8c944f848a60 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:52.014846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09c7ac-9550-4f92-8302-8c944f848a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:52.015145 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09c7ac-9550-4f92-8302-8c944f848a60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:52.028289 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09c7ac-9550-4f92-8302-8c944f848a60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:28:52.033815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-8c09c7ac-9550-4f92-8302-8c944f848a60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:28:52.033815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:28:52.033815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:28:52.033815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:28:52.033815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:28:52.033815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:28:52.033815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:28:52.033815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:28:52.033815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:28:52.033815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:28:52.033815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:28:52.033815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:28:52.033815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:28:52.033815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:28:52.033815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:28:52.033815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:28:52.034751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:28:52.034751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:28:52.034751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:28:52.034751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:28:52.034751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:28:52.034751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:28:52.034751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:28:52.034751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:28:52.034751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:28:52.034751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:28:52.034751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:28:52.034751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:28:52.034751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:28:52.034751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:28:52.034751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:28:52.034751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:28:52.035508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:28:52.035508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:28:52.035508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:28:52.035508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:28:52.035508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:28:52.035508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:28:52.035508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:28:52.035508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:28:52.035508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:28:52.035508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:28:52.035508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:28:52.035508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:28:52.035508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 35/68] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:28:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:28:52.040012 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09c7ac-9550-4f92-8302-8c944f848a60 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:52.085328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7387f-6985-4e21-8c42-5095792f302c None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:52.099728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:52.124135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4874f2-2f62-4ffb-ad93-59c96d350b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:52.125966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4874f2-2f62-4ffb-ad93-59c96d350b2d None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:52.126247 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4874f2-2f62-4ffb-ad93-59c96d350b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:52.126520 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4874f2-2f62-4ffb-ad93-59c96d350b2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:52.143696 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4874f2-2f62-4ffb-ad93-59c96d350b2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:28:52.149790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-9d4874f2-2f62-4ffb-ad93-59c96d350b2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:28:52.149790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:28:52.149790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:28:52.149790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:28:52.149790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:28:52.149790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:28:52.149790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:28:52.149790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:28:52.149790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:28:52.149790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:28:52.149790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:28:52.149790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:28:52.149790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:28:52.149790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:28:52.149790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:28:52.149790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:28:52.150726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:28:52.150726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:28:52.150726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:28:52.150726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:28:52.150726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:28:52.150726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:28:52.150726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:28:52.150726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:28:52.150726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:28:52.150726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:28:52.150726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:28:52.150726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:28:52.150726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:28:52.150726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:28:52.150726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:28:52.150726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:28:52.151487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:28:52.151487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:28:52.151487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:28:52.151487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:28:52.151487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:28:52.151487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:28:52.151487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:28:52.151487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:28:52.151487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:28:52.151487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:28:52.151487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:28:52.151487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:28:52.154632 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 34/69] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:28:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:28:52.160372 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4874f2-2f62-4ffb-ad93-59c96d350b2d None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:52.172086 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09c7ac-9550-4f92-8302-8c944f848a60 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:52.191038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:52.209409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccff69-fb06-4b76-8878-0d9269bd3b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:52.210059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccff69-fb06-4b76-8878-0d9269bd3b13 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:52.210343 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccff69-fb06-4b76-8878-0d9269bd3b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:52.210692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccff69-fb06-4b76-8878-0d9269bd3b13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:52.222317 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ccff69-fb06-4b76-8878-0d9269bd3b13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:28:52.227983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-d8ccff69-fb06-4b76-8878-0d9269bd3b13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:28:52.227983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:28:52.227983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:28:52.227983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:28:52.227983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:28:52.227983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:28:52.227983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:28:52.227983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:28:52.227983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:28:52.227983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:28:52.227983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:28:52.227983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:28:52.227983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:28:52.227983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:28:52.227983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:28:52.227983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:28:52.228838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:28:52.228838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:28:52.228838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:28:52.228838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:28:52.228838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:28:52.228838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:28:52.228838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:28:52.228838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:28:52.228838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:28:52.228838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:28:52.228838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:28:52.228838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:28:52.228838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:28:52.228838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:28:52.228838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:28:52.228838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:28:52.229649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:28:52.229649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:28:52.229649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:28:52.229649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:28:52.229649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:28:52.229649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:28:52.229649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:28:52.229649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:28:52.229649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:28:52.229649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:28:52.229649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:28:52.229649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:28:52.229649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 36/70] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:28:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:28:52.233606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c406f2b-e63c-4cc9-b82b-7c8717f9ff50 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:52.233606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c406f2b-e63c-4cc9-b82b-7c8717f9ff50 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:52.233606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c406f2b-e63c-4cc9-b82b-7c8717f9ff50 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:52.233606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 37/71] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:52.237627 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c406f2b-e63c-4cc9-b82b-7c8717f9ff50 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:52.277659 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4874f2-2f62-4ffb-ad93-59c96d350b2d None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:52.312860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc26ff-f7b4-4830-8c38-d70274b7a02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:52.313526 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc26ff-f7b4-4830-8c38-d70274b7a02c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:52.313802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc26ff-f7b4-4830-8c38-d70274b7a02c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:52.314076 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc26ff-f7b4-4830-8c38-d70274b7a02c None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:52.314748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc26ff-f7b4-4830-8c38-d70274b7a02c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:52.315053 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc26ff-f7b4-4830-8c38-d70274b7a02c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:52.316460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc26ff-f7b4-4830-8c38-d70274b7a02c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:52.325073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 35/72] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:28:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:52.326991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc26ff-f7b4-4830-8c38-d70274b7a02c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:52.337960 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c406f2b-e63c-4cc9-b82b-7c8717f9ff50 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:52.362568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:52.373587 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7b603-120c-4a23-ac31-7c4ea22e0622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:52.374259 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7b603-120c-4a23-ac31-7c4ea22e0622 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:52.374489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7b603-120c-4a23-ac31-7c4ea22e0622 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:52.374775 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7b603-120c-4a23-ac31-7c4ea22e0622 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:52.375441 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7b603-120c-4a23-ac31-7c4ea22e0622 None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:52.375748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7b603-120c-4a23-ac31-7c4ea22e0622 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:52.377174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7b603-120c-4a23-ac31-7c4ea22e0622 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:52.385393 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 38/73] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:28:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:52.390547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec97251-2955-4905-98a2-8063a81b8671 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:52.390787 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec97251-2955-4905-98a2-8063a81b8671 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:52.391064 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec97251-2955-4905-98a2-8063a81b8671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:52.409813 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-cec97251-2955-4905-98a2-8063a81b8671 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:52.411001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cec97251-2955-4905-98a2-8063a81b8671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:52.428931 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc26ff-f7b4-4830-8c38-d70274b7a02c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:52.442851 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:52.463114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d2363-e4f2-4cf0-86bf-4ad95c39eaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:52.463731 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d2363-e4f2-4cf0-86bf-4ad95c39eaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:52.464010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d2363-e4f2-4cf0-86bf-4ad95c39eaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:52.464281 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d2363-e4f2-4cf0-86bf-4ad95c39eaa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:52.464947 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d2363-e4f2-4cf0-86bf-4ad95c39eaa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:52.465283 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d2363-e4f2-4cf0-86bf-4ad95c39eaa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:52.466800 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d2363-e4f2-4cf0-86bf-4ad95c39eaa6 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:52.475654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 36/74] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:28:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:52.476658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f4184-0c45-4b47-b308-39204dde4775 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:52.476943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f4184-0c45-4b47-b308-39204dde4775 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:52.477250 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f4184-0c45-4b47-b308-39204dde4775 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:52.478435 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 37/75] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:52.480948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f4184-0c45-4b47-b308-39204dde4775 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:52.482289 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 39/76] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:28:52.486078 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec97251-2955-4905-98a2-8063a81b8671 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:52.579781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:52.583354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec97251-2955-4905-98a2-8063a81b8671 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:52.588583 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f4184-0c45-4b47-b308-39204dde4775 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:52.617791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473e5d0-4b14-4709-bfdf-d53fd37e9610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:52.618439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473e5d0-4b14-4709-bfdf-d53fd37e9610 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:52.618719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473e5d0-4b14-4709-bfdf-d53fd37e9610 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:52.618991 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473e5d0-4b14-4709-bfdf-d53fd37e9610 None admin] PATH_INFO: `/v3/users/93500b88e77e4d4ba8f347e33daa2e69` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:52.625360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741cdd7-8991-4732-ad3a-b6d9bbe82cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:52.625940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741cdd7-8991-4732-ad3a-b6d9bbe82cfd None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:52.626215 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741cdd7-8991-4732-ad3a-b6d9bbe82cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:52.626521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741cdd7-8991-4732-ad3a-b6d9bbe82cfd None admin] PATH_INFO: `/v3/users/93500b88e77e4d4ba8f347e33daa2e69` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:52.632944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473e5d0-4b14-4709-bfdf-d53fd37e9610 None admin] RBAC: Authorizing `identity:get_user(user_id=93500b88e77e4d4ba8f347e33daa2e69)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:52.634447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473e5d0-4b14-4709-bfdf-d53fd37e9610 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:52.636287 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 40/77] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:28:52 2022] GET /identity/v3/users/93500b88e77e4d4ba8f347e33daa2e69 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:52.638284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473e5d0-4b14-4709-bfdf-d53fd37e9610 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:52.645635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741cdd7-8991-4732-ad3a-b6d9bbe82cfd None admin] RBAC: Authorizing `identity:get_user(user_id=93500b88e77e4d4ba8f347e33daa2e69)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:52.647096 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741cdd7-8991-4732-ad3a-b6d9bbe82cfd None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:52.653364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 38/78] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:28:52 2022] GET /identity/v3/users/93500b88e77e4d4ba8f347e33daa2e69 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:52.653364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453118b-a311-4f6e-a8ef-1814960a4f94 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:52.653364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453118b-a311-4f6e-a8ef-1814960a4f94 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:52.653364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453118b-a311-4f6e-a8ef-1814960a4f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:52.671532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-5453118b-a311-4f6e-a8ef-1814960a4f94 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:52.672790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5453118b-a311-4f6e-a8ef-1814960a4f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:52.674951 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:52.752824 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473e5d0-4b14-4709-bfdf-d53fd37e9610 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:52.756009 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 39/79] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:28:52.756878 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453118b-a311-4f6e-a8ef-1814960a4f94 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:52.788769 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3246ea7-1152-469e-b7e3-3858d2ca7713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:52.789430 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3246ea7-1152-469e-b7e3-3858d2ca7713 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:52.789716 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3246ea7-1152-469e-b7e3-3858d2ca7713 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:52.789988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3246ea7-1152-469e-b7e3-3858d2ca7713 None admin] PATH_INFO: `/v3/users/7236c9ce10b9442581cf096a5b4cddf9` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:52.791148 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3246ea7-1152-469e-b7e3-3858d2ca7713 None admin] RBAC: Authorizing `identity:get_user(user_id=7236c9ce10b9442581cf096a5b4cddf9)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:52.792617 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3246ea7-1152-469e-b7e3-3858d2ca7713 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:52.794613 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 41/80] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:28:52 2022] GET /identity/v3/users/7236c9ce10b9442581cf096a5b4cddf9 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:52.799691 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3246ea7-1152-469e-b7e3-3858d2ca7713 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:52.849886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:52.867087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453118b-a311-4f6e-a8ef-1814960a4f94 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:52.898211 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:52.905530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3246ea7-1152-469e-b7e3-3858d2ca7713 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:52.905719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae932fc7-8ac6-4e42-942c-8144aa09ec3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:52.906433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae932fc7-8ac6-4e42-942c-8144aa09ec3c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:52.906721 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae932fc7-8ac6-4e42-942c-8144aa09ec3c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:52.907003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae932fc7-8ac6-4e42-942c-8144aa09ec3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:52.944108 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba017046-e60c-4e1c-96c9-3df084e27eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:52.944750 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba017046-e60c-4e1c-96c9-3df084e27eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:52.945033 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba017046-e60c-4e1c-96c9-3df084e27eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:52.945340 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba017046-e60c-4e1c-96c9-3df084e27eb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:52.981299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae932fc7-8ac6-4e42-942c-8144aa09ec3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:52.983273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae932fc7-8ac6-4e42-942c-8144aa09ec3c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:53.025192 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba017046-e60c-4e1c-96c9-3df084e27eb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:53.026742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba017046-e60c-4e1c-96c9-3df084e27eb0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:53.070588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:53.087710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 40/81] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:28:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:28:53.087710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae932fc7-8ac6-4e42-942c-8144aa09ec3c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:53.126103 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:53.134448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 42/82] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:28:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:28:53.135299 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba017046-e60c-4e1c-96c9-3df084e27eb0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:53.196185 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae932fc7-8ac6-4e42-942c-8144aa09ec3c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:53.233569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdb6fc-e0f7-4d1c-8735-22a0d81acf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:53.234350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdb6fc-e0f7-4d1c-8735-22a0d81acf0a None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:53.234627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdb6fc-e0f7-4d1c-8735-22a0d81acf0a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:53.235278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdb6fc-e0f7-4d1c-8735-22a0d81acf0a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:53.243401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba017046-e60c-4e1c-96c9-3df084e27eb0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:53.247480 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdb6fc-e0f7-4d1c-8735-22a0d81acf0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:28:53.252562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-12bdb6fc-e0f7-4d1c-8735-22a0d81acf0a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:28:53.252562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:28:53.252562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:28:53.252562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:28:53.252562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:28:53.252562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:28:53.252562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:28:53.252562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:28:53.252562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:28:53.252562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:28:53.252562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:28:53.252562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:28:53.252562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:28:53.252562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:28:53.252562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:28:53.252562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:28:53.253577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:28:53.253577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:28:53.253577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:28:53.253577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:28:53.253577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:28:53.253577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:28:53.253577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:28:53.253577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:28:53.253577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:28:53.253577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:28:53.253577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:28:53.253577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:28:53.253577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:28:53.253577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:28:53.253577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:28:53.253577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:28:53.254337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:28:53.254337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:28:53.254337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:28:53.254337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:28:53.254337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:28:53.254337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:28:53.254337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:28:53.254337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:28:53.254337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:28:53.254337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:28:53.254337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:28:53.254337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:28:53.254337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 41/83] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:28:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:28:53.255282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdb6fc-e0f7-4d1c-8735-22a0d81acf0a None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:53.283188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3317458-4624-4193-b5e6-269d03c5cb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:53.283828 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3317458-4624-4193-b5e6-269d03c5cb8e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:53.284115 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3317458-4624-4193-b5e6-269d03c5cb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:53.284393 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3317458-4624-4193-b5e6-269d03c5cb8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:53.303439 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:53.350231 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:53.358633 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3317458-4624-4193-b5e6-269d03c5cb8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:53.360244 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3317458-4624-4193-b5e6-269d03c5cb8e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:53.364503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdb6fc-e0f7-4d1c-8735-22a0d81acf0a None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:53.401870 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774046b-c873-4e02-a521-ae38961fe684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:53.402512 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774046b-c873-4e02-a521-ae38961fe684 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:53.402790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774046b-c873-4e02-a521-ae38961fe684 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:53.403065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774046b-c873-4e02-a521-ae38961fe684 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:53.404227 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3774046b-c873-4e02-a521-ae38961fe684 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:53.405730 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3774046b-c873-4e02-a521-ae38961fe684 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:53.407530 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 42/84] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:28:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:53.408443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774046b-c873-4e02-a521-ae38961fe684 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:53.467848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 43/85] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:28:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:28:53.468634 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3317458-4624-4193-b5e6-269d03c5cb8e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:53.528281 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774046b-c873-4e02-a521-ae38961fe684 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:53.542442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:53.564027 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895ffa6-81d5-4eb5-918d-e4316c0abea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:53.564676 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895ffa6-81d5-4eb5-918d-e4316c0abea4 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:53.564957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895ffa6-81d5-4eb5-918d-e4316c0abea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:53.565272 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895ffa6-81d5-4eb5-918d-e4316c0abea4 None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:53.566547 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895ffa6-81d5-4eb5-918d-e4316c0abea4 None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:53.568060 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895ffa6-81d5-4eb5-918d-e4316c0abea4 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:53.569771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 43/86] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:28:53 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:53.570543 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895ffa6-81d5-4eb5-918d-e4316c0abea4 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:53.574842 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:53.582763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3317458-4624-4193-b5e6-269d03c5cb8e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:53.618258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5ca4a-5e90-484e-a3dc-1809d5c624ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:53.618948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5ca4a-5e90-484e-a3dc-1809d5c624ed None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:53.619237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5ca4a-5e90-484e-a3dc-1809d5c624ed None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:53.619509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5ca4a-5e90-484e-a3dc-1809d5c624ed None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:53.620658 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5ca4a-5e90-484e-a3dc-1809d5c624ed None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:53.622143 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5ca4a-5e90-484e-a3dc-1809d5c624ed None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:53.623788 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 44/87] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:28:53 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:53.624537 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5ca4a-5e90-484e-a3dc-1809d5c624ed None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:53.676939 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895ffa6-81d5-4eb5-918d-e4316c0abea4 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:53.711993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbc0da-0966-4431-994c-5069b1772695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:53.712616 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbc0da-0966-4431-994c-5069b1772695 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:53.712899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbc0da-0966-4431-994c-5069b1772695 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:53.713202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbc0da-0966-4431-994c-5069b1772695 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:53.713881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbc0da-0966-4431-994c-5069b1772695 None admin] RBAC: Adding query filter params (name=member) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:53.714189 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbc0da-0966-4431-994c-5069b1772695 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:53.715749 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbc0da-0966-4431-994c-5069b1772695 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:53.724695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 44/88] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:28:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:53.724882 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5ca4a-5e90-484e-a3dc-1809d5c624ed None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:53.725500 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbc0da-0966-4431-994c-5069b1772695 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:53.764519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7788465-9d16-4058-81df-4ee6a73ba4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:53.765162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7788465-9d16-4058-81df-4ee6a73ba4b1 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:53.765442 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7788465-9d16-4058-81df-4ee6a73ba4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:53.765696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7788465-9d16-4058-81df-4ee6a73ba4b1 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:53.766359 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7788465-9d16-4058-81df-4ee6a73ba4b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:53.767795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7788465-9d16-4058-81df-4ee6a73ba4b1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:53.778797 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:53.795457 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 45/89] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:28:53 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:28:53.796343 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7788465-9d16-4058-81df-4ee6a73ba4b1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:53.823154 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:53.851107 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbc0da-0966-4431-994c-5069b1772695 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:53.893896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6c236-b8e9-48fd-95a4-0aa674310f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:53.894573 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6c236-b8e9-48fd-95a4-0aa674310f6f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:53.894846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6c236-b8e9-48fd-95a4-0aa674310f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:53.895132 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6c236-b8e9-48fd-95a4-0aa674310f6f None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:53.896339 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6c236-b8e9-48fd-95a4-0aa674310f6f None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:53.897874 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6c236-b8e9-48fd-95a4-0aa674310f6f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:53.904691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 45/90] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:28:53 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:53.904691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6c236-b8e9-48fd-95a4-0aa674310f6f None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:53.925080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7788465-9d16-4058-81df-4ee6a73ba4b1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:53.966642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e02ec1-8fef-439c-a262-d0f77fb4375d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:53.967277 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e02ec1-8fef-439c-a262-d0f77fb4375d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:53.967556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e02ec1-8fef-439c-a262-d0f77fb4375d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:53.967873 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e02ec1-8fef-439c-a262-d0f77fb4375d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:53.968656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e02ec1-8fef-439c-a262-d0f77fb4375d None admin] RBAC: Adding query filter params (role.id=bb454faf5dd643b892c6f9b670ad1f2d, scope.project.id=4e48d801558a426b8b994cb2bab00a20, user.id=93500b88e77e4d4ba8f347e33daa2e69) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:53.968971 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e02ec1-8fef-439c-a262-d0f77fb4375d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:53.970558 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e02ec1-8fef-439c-a262-d0f77fb4375d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:53.982219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 46/91] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:28:53 2022] GET /identity/v3/role_assignments?user.id=93500b88e77e4d4ba8f347e33daa2e69&scope.project.id=4e48d801558a426b8b994cb2bab00a20&role.id=bb454faf5dd643b892c6f9b670ad1f2d => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:53.982219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e02ec1-8fef-439c-a262-d0f77fb4375d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:54.036733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6c236-b8e9-48fd-95a4-0aa674310f6f None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:54.050621 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:54.094675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524092d4-e72a-4973-a086-02fab4627bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:54.095482 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524092d4-e72a-4973-a086-02fab4627bfc None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:54.095855 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524092d4-e72a-4973-a086-02fab4627bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:54.096182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524092d4-e72a-4973-a086-02fab4627bfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:54.096952 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524092d4-e72a-4973-a086-02fab4627bfc None admin] RBAC: Adding query filter params (role.id=05e78a4a009c41488cf3d2c47e8cbe30, scope.project.id=4e48d801558a426b8b994cb2bab00a20, user.id=93500b88e77e4d4ba8f347e33daa2e69) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:54.097523 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524092d4-e72a-4973-a086-02fab4627bfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:54.099267 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524092d4-e72a-4973-a086-02fab4627bfc None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:54.102053 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:54.107777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 46/92] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:28:53 2022] GET /identity/v3/role_assignments?user.id=93500b88e77e4d4ba8f347e33daa2e69&scope.project.id=4e48d801558a426b8b994cb2bab00a20&role.id=05e78a4a009c41488cf3d2c47e8cbe30 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:54.108679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524092d4-e72a-4973-a086-02fab4627bfc None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:54.126753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e02ec1-8fef-439c-a262-d0f77fb4375d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:54.171398 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fd50c-d167-4a49-a7ed-1e1d5ad7411d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:54.172467 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fd50c-d167-4a49-a7ed-1e1d5ad7411d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:54.173009 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fd50c-d167-4a49-a7ed-1e1d5ad7411d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:54.173378 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fd50c-d167-4a49-a7ed-1e1d5ad7411d None admin] PATH_INFO: `/v3/users/93500b88e77e4d4ba8f347e33daa2e69` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:54.174815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fd50c-d167-4a49-a7ed-1e1d5ad7411d None admin] RBAC: Authorizing `identity:get_user(user_id=93500b88e77e4d4ba8f347e33daa2e69)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:54.176447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fd50c-d167-4a49-a7ed-1e1d5ad7411d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:54.178782 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 47/93] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:28:53 2022] GET /identity/v3/users/93500b88e77e4d4ba8f347e33daa2e69 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:54.185638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fd50c-d167-4a49-a7ed-1e1d5ad7411d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:54.273605 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524092d4-e72a-4973-a086-02fab4627bfc None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:54.329432 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1fced-ecaa-45b3-a442-94725d8ac8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:54.330451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1fced-ecaa-45b3-a442-94725d8ac8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:54.330923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1fced-ecaa-45b3-a442-94725d8ac8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:54.331410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1fced-ecaa-45b3-a442-94725d8ac8c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:54.332611 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1fced-ecaa-45b3-a442-94725d8ac8c5 None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=4e48d801558a426b8b994cb2bab00a20, user.id=7236c9ce10b9442581cf096a5b4cddf9) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:54.333222 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1fced-ecaa-45b3-a442-94725d8ac8c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:54.334951 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1fced-ecaa-45b3-a442-94725d8ac8c5 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:54.342113 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 47/94] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:28:54 2022] GET /identity/v3/role_assignments?user.id=7236c9ce10b9442581cf096a5b4cddf9&scope.project.id=4e48d801558a426b8b994cb2bab00a20&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:54.343831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fd50c-d167-4a49-a7ed-1e1d5ad7411d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:54.388811 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796f390-aeea-46b3-a87e-26f361506eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:54.389499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796f390-aeea-46b3-a87e-26f361506eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:54.389778 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796f390-aeea-46b3-a87e-26f361506eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:54.390052 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796f390-aeea-46b3-a87e-26f361506eb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:54.413552 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:54.473844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796f390-aeea-46b3-a87e-26f361506eb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:54.476289 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796f390-aeea-46b3-a87e-26f361506eb5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:54.596489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 48/95] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:28:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:28:54.603248 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1fced-ecaa-45b3-a442-94725d8ac8c5 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:54.667204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:54.741693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1fced-ecaa-45b3-a442-94725d8ac8c5 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:54.784260 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f52ed4-6faa-4303-90f1-5bdf8f839ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:54.784919 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f52ed4-6faa-4303-90f1-5bdf8f839ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:54.785219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f52ed4-6faa-4303-90f1-5bdf8f839ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:54.785494 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f52ed4-6faa-4303-90f1-5bdf8f839ca1 None admin] PATH_INFO: `/v3/projects/f2e8bda5cd0745bbb7a1fd43ceeadc7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:54.786647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f52ed4-6faa-4303-90f1-5bdf8f839ca1 None admin] RBAC: Authorizing `identity:get_project(project_id=f2e8bda5cd0745bbb7a1fd43ceeadc7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:54.788105 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f52ed4-6faa-4303-90f1-5bdf8f839ca1 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:54.790122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 48/96] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:28:54 2022] GET /identity/v3/projects/f2e8bda5cd0745bbb7a1fd43ceeadc7a => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:54.795968 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796f390-aeea-46b3-a87e-26f361506eb5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:54.878565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:54.944274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796f390-aeea-46b3-a87e-26f361506eb5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:54.987326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3af615-3561-4d00-814d-112f01791359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:54.988324 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3af615-3561-4d00-814d-112f01791359 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:54.988800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3af615-3561-4d00-814d-112f01791359 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:54.989354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3af615-3561-4d00-814d-112f01791359 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:54.990303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3af615-3561-4d00-814d-112f01791359 None admin] RBAC: Adding query filter params (role.id=bb454faf5dd643b892c6f9b670ad1f2d, scope.project.id=f2e8bda5cd0745bbb7a1fd43ceeadc7a, user.id=93500b88e77e4d4ba8f347e33daa2e69) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:54.990816 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3af615-3561-4d00-814d-112f01791359 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:54.992499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3af615-3561-4d00-814d-112f01791359 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:55.003467 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 49/97] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:28:54 2022] GET /identity/v3/role_assignments?user.id=93500b88e77e4d4ba8f347e33daa2e69&scope.project.id=f2e8bda5cd0745bbb7a1fd43ceeadc7a&role.id=bb454faf5dd643b892c6f9b670ad1f2d => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:57.154505 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4ed14-8f87-4f44-9a82-2ebdf6750657 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:57.155038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4ed14-8f87-4f44-9a82-2ebdf6750657 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:57.155145 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4ed14-8f87-4f44-9a82-2ebdf6750657 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:57.156714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 49/98] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:57.162586 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a8bc2-2a4f-4b26-9479-1a2274dd7f0a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:57.163339 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a8bc2-2a4f-4b26-9479-1a2274dd7f0a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:57.163339 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a8bc2-2a4f-4b26-9479-1a2274dd7f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:57.191310 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-923a8bc2-2a4f-4b26-9479-1a2274dd7f0a None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:57.192544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-923a8bc2-2a4f-4b26-9479-1a2274dd7f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:57.291922 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 50/99] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:28:57.518196 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0972d8c-c485-4d10-b2ad-1466e7d08207 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:57.518376 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0972d8c-c485-4d10-b2ad-1466e7d08207 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:57.518639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0972d8c-c485-4d10-b2ad-1466e7d08207 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:57.520003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 50/100] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:57.526071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5af8e7-d442-411d-8aa9-7797ceeab629 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:57.526372 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5af8e7-d442-411d-8aa9-7797ceeab629 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:57.526630 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5af8e7-d442-411d-8aa9-7797ceeab629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:57.550607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b62b1-7e94-4e99-ad31-60cc31643eea None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:57.550932 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b62b1-7e94-4e99-ad31-60cc31643eea None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:57.551146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b62b1-7e94-4e99-ad31-60cc31643eea None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:57.552497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 51/101] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:28:57.554560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-0b5af8e7-d442-411d-8aa9-7797ceeab629 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:57.555804 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0b5af8e7-d442-411d-8aa9-7797ceeab629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:57.563037 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b274207-f93e-462a-bb29-d10a891a47f4 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:57.563376 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b274207-f93e-462a-bb29-d10a891a47f4 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:57.563577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b274207-f93e-462a-bb29-d10a891a47f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:57.595798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-2b274207-f93e-462a-bb29-d10a891a47f4 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:57.597021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2b274207-f93e-462a-bb29-d10a891a47f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:57.657830 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:57.716030 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 51/102] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:28:57.717513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33c7be-1559-4b6c-a56e-7ec31bbdd4a8 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:57.717800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33c7be-1559-4b6c-a56e-7ec31bbdd4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:57.718077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33c7be-1559-4b6c-a56e-7ec31bbdd4a8 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:57.719403 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 52/103] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:57.720370 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 52/104] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:28:57.724939 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33c7be-1559-4b6c-a56e-7ec31bbdd4a8 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:57.735424 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177cf6e-47fd-4a9c-8fed-d10945c3ef6c None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:57.736046 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177cf6e-47fd-4a9c-8fed-d10945c3ef6c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:57.736516 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177cf6e-47fd-4a9c-8fed-d10945c3ef6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:57.773784 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-8177cf6e-47fd-4a9c-8fed-d10945c3ef6c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:57.775529 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-8177cf6e-47fd-4a9c-8fed-d10945c3ef6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:57.839624 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:57.910017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 53/105] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:28:57.911249 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e010d6-87c7-43fc-a26d-d2bd1724d857 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:57.911696 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e010d6-87c7-43fc-a26d-d2bd1724d857 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:57.912119 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e010d6-87c7-43fc-a26d-d2bd1724d857 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:57.913669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 54/106] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:57.920799 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2c600-16c1-40a7-be97-b428c9d82312 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:57.921346 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2c600-16c1-40a7-be97-b428c9d82312 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:57.921762 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2c600-16c1-40a7-be97-b428c9d82312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:57.950607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33c7be-1559-4b6c-a56e-7ec31bbdd4a8 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:57.953617 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-79f2c600-16c1-40a7-be97-b428c9d82312 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:57.955841 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-79f2c600-16c1-40a7-be97-b428c9d82312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:58.015258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c01f4-a5cf-4693-8f86-50a738899a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:58.016174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c01f4-a5cf-4693-8f86-50a738899a3f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:58.016784 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c01f4-a5cf-4693-8f86-50a738899a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:58.017240 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c01f4-a5cf-4693-8f86-50a738899a3f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:58.029841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c01f4-a5cf-4693-8f86-50a738899a3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:28:58.035425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-f66c01f4-a5cf-4693-8f86-50a738899a3f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:28:58.035425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:28:58.035425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:28:58.035425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:28:58.035425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:28:58.035425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:28:58.035425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:28:58.035425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:28:58.035425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:28:58.035425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:28:58.035425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:28:58.035425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:28:58.035425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:28:58.035425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:28:58.035425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:28:58.035425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:28:58.036344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:28:58.036344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:28:58.036344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:28:58.036344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:28:58.036344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:28:58.036344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:28:58.036344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:28:58.036344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:28:58.036344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:28:58.036344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:28:58.036344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:28:58.036344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:28:58.036344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:28:58.036344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:28:58.036344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:28:58.036344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:28:58.037158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:28:58.037158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:28:58.037158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:28:58.037158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:28:58.037158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:28:58.037158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:28:58.037158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:28:58.037158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:28:58.037158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:28:58.037158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:28:58.037158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:28:58.037158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:28:58.040058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 53/107] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:28:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:28:58.045887 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c01f4-a5cf-4693-8f86-50a738899a3f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:58.089521 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:58.097321 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 55/108] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:28:58.097321 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2c600-16c1-40a7-be97-b428c9d82312 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:58.267091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c01f4-a5cf-4693-8f86-50a738899a3f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:58.282014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:58.284830 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2c600-16c1-40a7-be97-b428c9d82312 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:58.334804 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe46e0-a4f5-45e1-a240-baf78cfd8622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:58.335789 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe46e0-a4f5-45e1-a240-baf78cfd8622 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:58.337673 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe46e0-a4f5-45e1-a240-baf78cfd8622 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:58.338175 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe46e0-a4f5-45e1-a240-baf78cfd8622 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:58.339282 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe46e0-a4f5-45e1-a240-baf78cfd8622 None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:58.341064 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe46e0-a4f5-45e1-a240-baf78cfd8622 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:58.343705 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe46e0-a4f5-45e1-a240-baf78cfd8622 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:58.353600 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6c8e4-e167-4735-b944-1e07a6a2347f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:58.356966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6c8e4-e167-4735-b944-1e07a6a2347f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:58.357454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6c8e4-e167-4735-b944-1e07a6a2347f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:58.357895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6c8e4-e167-4735-b944-1e07a6a2347f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:58.363484 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 54/109] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:28:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:58.365829 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe46e0-a4f5-45e1-a240-baf78cfd8622 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:58.376597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6c8e4-e167-4735-b944-1e07a6a2347f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:28:58.382804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-92c6c8e4-e167-4735-b944-1e07a6a2347f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:28:58.382804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:28:58.382804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:28:58.382804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:28:58.382804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:28:58.382804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:28:58.382804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:28:58.382804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:28:58.382804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:28:58.382804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:28:58.382804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:28:58.382804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:28:58.382804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:28:58.382804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:28:58.382804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:28:58.382804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:28:58.383654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:28:58.383654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:28:58.383654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:28:58.383654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:28:58.383654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:28:58.383654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:28:58.383654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:28:58.383654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:28:58.383654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:28:58.383654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:28:58.383654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:28:58.383654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:28:58.383654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:28:58.383654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:28:58.383654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:28:58.383654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:28:58.384443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:28:58.384443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:28:58.384443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:28:58.384443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:28:58.384443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:28:58.384443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:28:58.384443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:28:58.384443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:28:58.384443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:28:58.384443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:28:58.384443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:28:58.384443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:28:58.387004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 56/110] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:28:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:28:58.392296 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6c8e4-e167-4735-b944-1e07a6a2347f None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:58.479480 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:58.487852 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe46e0-a4f5-45e1-a240-baf78cfd8622 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:58.511284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6c8e4-e167-4735-b944-1e07a6a2347f None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:58.530810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1b606-658b-4ab8-af25-ee0a4d331d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:58.531469 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1b606-658b-4ab8-af25-ee0a4d331d15 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:58.531757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1b606-658b-4ab8-af25-ee0a4d331d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:58.532096 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1b606-658b-4ab8-af25-ee0a4d331d15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:58.545342 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1b606-658b-4ab8-af25-ee0a4d331d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:28:58.551907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-c8c1b606-658b-4ab8-af25-ee0a4d331d15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:28:58.551907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:28:58.551907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:28:58.551907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:28:58.551907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:28:58.551907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:28:58.551907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:28:58.551907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:28:58.551907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:28:58.551907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:28:58.551907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:28:58.551907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:28:58.551907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:28:58.551907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:28:58.551907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:28:58.551907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:28:58.552800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:28:58.552800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:28:58.552800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:28:58.552800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:28:58.552800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:28:58.552800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:28:58.552800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:28:58.552800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:28:58.552800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:28:58.552800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:28:58.552800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:28:58.552800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:28:58.552800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:28:58.552800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:28:58.552800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:28:58.552800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:28:58.553584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:28:58.553584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:28:58.553584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:28:58.553584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:28:58.553584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:28:58.553584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:28:58.553584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:28:58.553584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:28:58.553584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:28:58.553584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:28:58.553584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:28:58.553584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:28:58.557216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08ace7-5efc-4805-8a9b-5f814ebbe996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:58.557706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 55/111] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:28:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:28:58.558473 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08ace7-5efc-4805-8a9b-5f814ebbe996 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:58.558714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1b606-658b-4ab8-af25-ee0a4d331d15 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:58.559233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08ace7-5efc-4805-8a9b-5f814ebbe996 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:58.559792 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08ace7-5efc-4805-8a9b-5f814ebbe996 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:58.572423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b08ace7-5efc-4805-8a9b-5f814ebbe996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:28:58.578525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-7b08ace7-5efc-4805-8a9b-5f814ebbe996 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:28:58.578525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:28:58.578525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:28:58.578525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:28:58.578525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:28:58.578525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:28:58.578525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:28:58.578525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:28:58.578525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:28:58.578525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:28:58.578525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:28:58.578525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:28:58.578525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:28:58.578525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:28:58.578525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:28:58.578525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:28:58.579428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:28:58.579428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:28:58.579428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:28:58.579428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:28:58.579428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:28:58.579428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:28:58.579428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:28:58.579428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:28:58.579428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:28:58.579428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:28:58.579428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:28:58.579428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:28:58.579428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:28:58.579428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:28:58.579428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:28:58.579428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:28:58.580170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:28:58.580170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:28:58.580170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:28:58.580170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:28:58.580170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:28:58.580170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:28:58.580170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:28:58.580170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:28:58.580170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:28:58.580170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:28:58.580170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:28:58.580170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:28:58.582664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 57/112] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:28:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:28:58.587543 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08ace7-5efc-4805-8a9b-5f814ebbe996 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:58.612539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:58.680309 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1b606-658b-4ab8-af25-ee0a4d331d15 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:58.706046 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08ace7-5efc-4805-8a9b-5f814ebbe996 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:58.718277 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a54b6-d3ab-4aaf-8beb-70ec8f666be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:58.718924 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a54b6-d3ab-4aaf-8beb-70ec8f666be3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:58.719203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a54b6-d3ab-4aaf-8beb-70ec8f666be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:58.719474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a54b6-d3ab-4aaf-8beb-70ec8f666be3 None admin] PATH_INFO: `/v3/users/93500b88e77e4d4ba8f347e33daa2e69` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:58.720624 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a54b6-d3ab-4aaf-8beb-70ec8f666be3 None admin] RBAC: Authorizing `identity:get_user(user_id=93500b88e77e4d4ba8f347e33daa2e69)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:58.722155 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a54b6-d3ab-4aaf-8beb-70ec8f666be3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:58.724178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 56/113] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:28:58 2022] GET /identity/v3/users/93500b88e77e4d4ba8f347e33daa2e69 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:58.725004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a54b6-d3ab-4aaf-8beb-70ec8f666be3 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:58.748857 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b6e31-77b9-4c7b-88b7-36e18f89654f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:58.749637 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6e31-77b9-4c7b-88b7-36e18f89654f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:58.750049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6e31-77b9-4c7b-88b7-36e18f89654f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:58.750365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6e31-77b9-4c7b-88b7-36e18f89654f None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:58.751267 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b6e31-77b9-4c7b-88b7-36e18f89654f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:58.751622 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b6e31-77b9-4c7b-88b7-36e18f89654f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:58.753128 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b6e31-77b9-4c7b-88b7-36e18f89654f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:58.757492 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:58.764022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 58/114] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:28:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:58.764895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b6e31-77b9-4c7b-88b7-36e18f89654f None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:58.837102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a54b6-d3ab-4aaf-8beb-70ec8f666be3 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:58.847763 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:58.873628 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75a426-98a3-4d66-b01f-c462e8427552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:58.874257 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75a426-98a3-4d66-b01f-c462e8427552 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:58.874537 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75a426-98a3-4d66-b01f-c462e8427552 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:58.874810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75a426-98a3-4d66-b01f-c462e8427552 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:58.874904 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b6e31-77b9-4c7b-88b7-36e18f89654f None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:58.875483 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75a426-98a3-4d66-b01f-c462e8427552 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:58.875783 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75a426-98a3-4d66-b01f-c462e8427552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:58.877209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75a426-98a3-4d66-b01f-c462e8427552 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:58.886427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 57/115] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:28:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:58.886427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1aa5d-1bcd-4d60-a7b3-07cf90027fc7 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:58.886427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1aa5d-1bcd-4d60-a7b3-07cf90027fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:58.886427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1aa5d-1bcd-4d60-a7b3-07cf90027fc7 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:58.886427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 58/116] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:28:58.886427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1aa5d-1bcd-4d60-a7b3-07cf90027fc7 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:58.920158 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3f101-750b-44d4-9054-3347a397cb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:58.920780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3f101-750b-44d4-9054-3347a397cb09 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:58.921060 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3f101-750b-44d4-9054-3347a397cb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:58.921374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3f101-750b-44d4-9054-3347a397cb09 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:58.922050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3f101-750b-44d4-9054-3347a397cb09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:28:58.922354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3f101-750b-44d4-9054-3347a397cb09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:58.923740 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3f101-750b-44d4-9054-3347a397cb09 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:58.932675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 59/117] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:28:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:58.933956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3f101-750b-44d4-9054-3347a397cb09 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:58.989870 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:59.009578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1aa5d-1bcd-4d60-a7b3-07cf90027fc7 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:59.050464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3f101-750b-44d4-9054-3347a397cb09 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:59.050702 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa15263-6765-4e9e-8625-400a1b988eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:59.051246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa15263-6765-4e9e-8625-400a1b988eda None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:59.051526 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa15263-6765-4e9e-8625-400a1b988eda None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:59.051798 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa15263-6765-4e9e-8625-400a1b988eda None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:59.052953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa15263-6765-4e9e-8625-400a1b988eda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:59.054456 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa15263-6765-4e9e-8625-400a1b988eda None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:59.056270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 59/118] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:28:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:59.056860 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa15263-6765-4e9e-8625-400a1b988eda None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:59.088732 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e48f02-3e31-4231-9890-bddbc87c0a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:59.089684 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e48f02-3e31-4231-9890-bddbc87c0a1e None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:59.090173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e48f02-3e31-4231-9890-bddbc87c0a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:59.090551 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e48f02-3e31-4231-9890-bddbc87c0a1e None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:59.092425 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e48f02-3e31-4231-9890-bddbc87c0a1e None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:59.094952 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e48f02-3e31-4231-9890-bddbc87c0a1e None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:59.097854 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 60/119] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:28:58 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:59.099583 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e48f02-3e31-4231-9890-bddbc87c0a1e None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:59.116157 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:59.184070 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa15263-6765-4e9e-8625-400a1b988eda None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:59.220449 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e48f02-3e31-4231-9890-bddbc87c0a1e None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:59.224124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d90f4d-a6db-4a32-81a9-994cd6fb0fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:59.224869 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d90f4d-a6db-4a32-81a9-994cd6fb0fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:59.225293 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d90f4d-a6db-4a32-81a9-994cd6fb0fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:59.225674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d90f4d-a6db-4a32-81a9-994cd6fb0fe3 None admin] PATH_INFO: `/v3/users/83569985bd5a40ccbdefde7c5a100678` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:59.243406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d90f4d-a6db-4a32-81a9-994cd6fb0fe3 None admin] RBAC: Authorizing `identity:get_user(user_id=83569985bd5a40ccbdefde7c5a100678)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:59.245260 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d90f4d-a6db-4a32-81a9-994cd6fb0fe3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:59.249181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 60/120] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:28:59 2022] GET /identity/v3/users/83569985bd5a40ccbdefde7c5a100678 => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:59.249181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ff855-138c-4c51-9f3b-d8602f7f2638 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:59.249414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ff855-138c-4c51-9f3b-d8602f7f2638 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:59.249602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ff855-138c-4c51-9f3b-d8602f7f2638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:59.263037 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c882a0b-2b31-4295-ac52-e293c11ea2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:59.263692 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c882a0b-2b31-4295-ac52-e293c11ea2eb None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:59.263973 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c882a0b-2b31-4295-ac52-e293c11ea2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:59.264245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c882a0b-2b31-4295-ac52-e293c11ea2eb None admin] PATH_INFO: `/v3/users/93500b88e77e4d4ba8f347e33daa2e69` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:59.265481 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c882a0b-2b31-4295-ac52-e293c11ea2eb None admin] RBAC: Authorizing `identity:get_user(user_id=93500b88e77e4d4ba8f347e33daa2e69)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:59.266965 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c882a0b-2b31-4295-ac52-e293c11ea2eb None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:59.268448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-544ff855-138c-4c51-9f3b-d8602f7f2638 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:28:59.269310 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 61/121] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:28:59 2022] GET /identity/v3/users/93500b88e77e4d4ba8f347e33daa2e69 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:59.269460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c882a0b-2b31-4295-ac52-e293c11ea2eb None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:59.269763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-544ff855-138c-4c51-9f3b-d8602f7f2638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:28:59.278759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:59.342881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 61/122] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:28:59.343626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ff855-138c-4c51-9f3b-d8602f7f2638 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:59.358251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:59.396309 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c882a0b-2b31-4295-ac52-e293c11ea2eb None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:59.450126 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820eaa4-dc5a-48f4-8e16-67c904baf740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:59.451189 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820eaa4-dc5a-48f4-8e16-67c904baf740 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:59.451650 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820eaa4-dc5a-48f4-8e16-67c904baf740 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:59.452113 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820eaa4-dc5a-48f4-8e16-67c904baf740 None admin] PATH_INFO: `/v3/users/7236c9ce10b9442581cf096a5b4cddf9` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:59.454017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820eaa4-dc5a-48f4-8e16-67c904baf740 None admin] RBAC: Authorizing `identity:get_user(user_id=7236c9ce10b9442581cf096a5b4cddf9)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:59.456280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820eaa4-dc5a-48f4-8e16-67c904baf740 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:59.458900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 62/123] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:28:59 2022] GET /identity/v3/users/7236c9ce10b9442581cf096a5b4cddf9 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:28:59.462367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820eaa4-dc5a-48f4-8e16-67c904baf740 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:59.486734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ff855-138c-4c51-9f3b-d8602f7f2638 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:59.543136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac565d9b-ba80-48b9-a002-19443215f047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:59.544169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac565d9b-ba80-48b9-a002-19443215f047 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:59.544632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac565d9b-ba80-48b9-a002-19443215f047 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:59.545092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac565d9b-ba80-48b9-a002-19443215f047 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:59.546228 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac565d9b-ba80-48b9-a002-19443215f047 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:59.548588 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac565d9b-ba80-48b9-a002-19443215f047 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:59.571788 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:59.590100 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 62/124] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 26 05:28:59 2022] POST /identity/v3/projects => generated 302 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:28:59.598764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac565d9b-ba80-48b9-a002-19443215f047 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:59.634112 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820eaa4-dc5a-48f4-8e16-67c904baf740 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:59.686295 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be69c70-e452-4142-9952-c6040416da11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:59.687333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be69c70-e452-4142-9952-c6040416da11 None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:59.687819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be69c70-e452-4142-9952-c6040416da11 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:59.688235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be69c70-e452-4142-9952-c6040416da11 None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20/users/93500b88e77e4d4ba8f347e33daa2e69/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:59.691782 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be69c70-e452-4142-9952-c6040416da11 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e48d801558a426b8b994cb2bab00a20, user_id=93500b88e77e4d4ba8f347e33daa2e69, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:28:59.693772 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be69c70-e452-4142-9952-c6040416da11 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:28:59.712164 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 63/125] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:28:59 2022] PUT /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20/users/93500b88e77e4d4ba8f347e33daa2e69/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:28:59.729192 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be69c70-e452-4142-9952-c6040416da11 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:28:59.736324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:28:59.785866 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac565d9b-ba80-48b9-a002-19443215f047 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:28:59.858986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210be491-2893-45c8-89bb-417480fb244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:28:59.860046 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210be491-2893-45c8-89bb-417480fb244e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:28:59.860520 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210be491-2893-45c8-89bb-417480fb244e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:28:59.860989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210be491-2893-45c8-89bb-417480fb244e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:28:59.939992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be69c70-e452-4142-9952-c6040416da11 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:00.001490 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:00.010574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b612fb0-ad94-42db-9b7b-e971683297a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:00.012434 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b612fb0-ad94-42db-9b7b-e971683297a5 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:00.013752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b612fb0-ad94-42db-9b7b-e971683297a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:00.014316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b612fb0-ad94-42db-9b7b-e971683297a5 None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:00.018027 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b612fb0-ad94-42db-9b7b-e971683297a5 None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:00.018165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210be491-2893-45c8-89bb-417480fb244e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:00.019708 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b612fb0-ad94-42db-9b7b-e971683297a5 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:00.020456 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210be491-2893-45c8-89bb-417480fb244e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:00.022631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 64/126] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:28:59 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:00.023928 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b612fb0-ad94-42db-9b7b-e971683297a5 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:00.161775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:00.228338 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b612fb0-ad94-42db-9b7b-e971683297a5 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:00.248548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 63/127] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:00.250085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210be491-2893-45c8-89bb-417480fb244e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:00.322416 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb9414-e47b-4d0f-9a48-126a68c42519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:00.323553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb9414-e47b-4d0f-9a48-126a68c42519 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:00.324036 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb9414-e47b-4d0f-9a48-126a68c42519 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:00.324523 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb9414-e47b-4d0f-9a48-126a68c42519 None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:00.326765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb9414-e47b-4d0f-9a48-126a68c42519 None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:00.329632 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb9414-e47b-4d0f-9a48-126a68c42519 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:00.332635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 65/128] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:00 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:00.334109 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb9414-e47b-4d0f-9a48-126a68c42519 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:00.444689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:00.454096 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210be491-2893-45c8-89bb-417480fb244e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:00.498670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3aa3e2-a88c-46c0-b593-c307dfd90840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:00.500346 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3aa3e2-a88c-46c0-b593-c307dfd90840 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:00.500783 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3aa3e2-a88c-46c0-b593-c307dfd90840 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:00.501232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3aa3e2-a88c-46c0-b593-c307dfd90840 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:00.515980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3aa3e2-a88c-46c0-b593-c307dfd90840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:00.522146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-2c3aa3e2-a88c-46c0-b593-c307dfd90840 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:00.522146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:00.522146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:00.522146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:00.522146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:00.522146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:00.522146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:00.522146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:00.522146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:00.522146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:00.522146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:00.522146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:00.522146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:00.522146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:00.522146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:00.522146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:00.523318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:00.523318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:00.523318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:00.523318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:00.523318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:00.523318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:00.523318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:00.523318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:00.523318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:00.523318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:00.523318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:00.523318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:00.523318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:00.523318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:00.523318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:00.523318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:00.524374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:00.524374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:00.524374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:00.524374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:00.524374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:00.524374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:00.524374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:00.524374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:00.524374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:00.524374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:00.524374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:00.524374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:00.527379 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 64/129] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:00.528621 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3aa3e2-a88c-46c0-b593-c307dfd90840 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:00.561717 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb9414-e47b-4d0f-9a48-126a68c42519 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:00.635940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190f708-e3b0-4716-bfdf-3c791ffe1c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:00.637280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190f708-e3b0-4716-bfdf-3c791ffe1c7f None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:00.638366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190f708-e3b0-4716-bfdf-3c791ffe1c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:00.639452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190f708-e3b0-4716-bfdf-3c791ffe1c7f None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20/users/93500b88e77e4d4ba8f347e33daa2e69/roles/05e78a4a009c41488cf3d2c47e8cbe30` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:00.643771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7190f708-e3b0-4716-bfdf-3c791ffe1c7f None admin] RBAC: Authorizing `identity:create_grant(project_id=4e48d801558a426b8b994cb2bab00a20, user_id=93500b88e77e4d4ba8f347e33daa2e69, role_id=05e78a4a009c41488cf3d2c47e8cbe30)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:00.647306 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7190f708-e3b0-4716-bfdf-3c791ffe1c7f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:00.667395 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:00.668776 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3aa3e2-a88c-46c0-b593-c307dfd90840 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:00.681226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 66/130] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:29:00 2022] PUT /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20/users/93500b88e77e4d4ba8f347e33daa2e69/roles/05e78a4a009c41488cf3d2c47e8cbe30 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:00.686517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190f708-e3b0-4716-bfdf-3c791ffe1c7f None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:00.772781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843f1ab-559c-4f9e-99a6-12c342a33a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:00.775248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843f1ab-559c-4f9e-99a6-12c342a33a13 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:00.777823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843f1ab-559c-4f9e-99a6-12c342a33a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:00.780748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843f1ab-559c-4f9e-99a6-12c342a33a13 None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:00.785876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843f1ab-559c-4f9e-99a6-12c342a33a13 None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:00.790060 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843f1ab-559c-4f9e-99a6-12c342a33a13 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:00.800175 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 65/131] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:00 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:00.806302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843f1ab-559c-4f9e-99a6-12c342a33a13 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:00.864655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:00.955664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190f708-e3b0-4716-bfdf-3c791ffe1c7f None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:01.032512 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843f1ab-559c-4f9e-99a6-12c342a33a13 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:01.040863 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b66c2b-ca95-4382-8358-589ed055798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:01.043243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b66c2b-ca95-4382-8358-589ed055798a None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:01.044648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b66c2b-ca95-4382-8358-589ed055798a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:01.045496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b66c2b-ca95-4382-8358-589ed055798a None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20/users/7236c9ce10b9442581cf096a5b4cddf9/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:01.051086 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b66c2b-ca95-4382-8358-589ed055798a None admin] RBAC: Authorizing `identity:create_grant(project_id=4e48d801558a426b8b994cb2bab00a20, user_id=7236c9ce10b9442581cf096a5b4cddf9, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:01.055406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b66c2b-ca95-4382-8358-589ed055798a None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:01.080873 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570817d0-2b61-42a6-85eb-0582126e1bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:01.081703 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570817d0-2b61-42a6-85eb-0582126e1bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:01.082002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570817d0-2b61-42a6-85eb-0582126e1bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:01.082372 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570817d0-2b61-42a6-85eb-0582126e1bc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:01.083164 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570817d0-2b61-42a6-85eb-0582126e1bc0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:01.083477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570817d0-2b61-42a6-85eb-0582126e1bc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:01.085046 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570817d0-2b61-42a6-85eb-0582126e1bc0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:01.089733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 67/132] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:29:00 2022] PUT /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20/users/7236c9ce10b9442581cf096a5b4cddf9/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:01.097616 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b66c2b-ca95-4382-8358-589ed055798a None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:01.102697 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 66/133] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:01.112038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570817d0-2b61-42a6-85eb-0582126e1bc0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:01.243951 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:01.291677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:01.300735 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b66c2b-ca95-4382-8358-589ed055798a None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:01.325973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570817d0-2b61-42a6-85eb-0582126e1bc0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:01.355642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08d3b5-e302-4974-a1da-0f93299527eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:01.356445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08d3b5-e302-4974-a1da-0f93299527eb None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:01.356733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08d3b5-e302-4974-a1da-0f93299527eb None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:01.357014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08d3b5-e302-4974-a1da-0f93299527eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:01.357792 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08d3b5-e302-4974-a1da-0f93299527eb None admin] RBAC: Adding query filter params (role.id=27a25b7936b8463c9e6d34d2cb49e61f, scope.project.id=4e48d801558a426b8b994cb2bab00a20, user.id=83569985bd5a40ccbdefde7c5a100678) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:01.358122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08d3b5-e302-4974-a1da-0f93299527eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:01.359737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08d3b5-e302-4974-a1da-0f93299527eb None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:01.370642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8cbf4-ed63-4e31-8d08-27f06ccae6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:01.371756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8cbf4-ed63-4e31-8d08-27f06ccae6ee None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:01.372327 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8cbf4-ed63-4e31-8d08-27f06ccae6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:01.372798 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8cbf4-ed63-4e31-8d08-27f06ccae6ee None admin] PATH_INFO: `/v3/users/93500b88e77e4d4ba8f347e33daa2e69` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:01.374800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8cbf4-ed63-4e31-8d08-27f06ccae6ee None admin] RBAC: Authorizing `identity:get_user(user_id=93500b88e77e4d4ba8f347e33daa2e69)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:01.376454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 68/134] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:29:01 2022] GET /identity/v3/role_assignments?user.id=83569985bd5a40ccbdefde7c5a100678&scope.project.id=4e48d801558a426b8b994cb2bab00a20&role.id=27a25b7936b8463c9e6d34d2cb49e61f => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:01.377401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8cbf4-ed63-4e31-8d08-27f06ccae6ee None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:01.379516 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 67/135] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:01 2022] GET /identity/v3/users/93500b88e77e4d4ba8f347e33daa2e69 => generated 275 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:01.388538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08d3b5-e302-4974-a1da-0f93299527eb None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:01.539518 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08d3b5-e302-4974-a1da-0f93299527eb None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:01.566848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:01.590431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470e855-7b64-4b6f-9579-ad0140042890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:01.591647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470e855-7b64-4b6f-9579-ad0140042890 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:01.592158 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470e855-7b64-4b6f-9579-ad0140042890 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:01.592668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470e855-7b64-4b6f-9579-ad0140042890 None admin] PATH_INFO: `/v3/projects/f2e8bda5cd0745bbb7a1fd43ceeadc7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:01.594861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470e855-7b64-4b6f-9579-ad0140042890 None admin] RBAC: Authorizing `identity:get_project(project_id=f2e8bda5cd0745bbb7a1fd43ceeadc7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:01.597706 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470e855-7b64-4b6f-9579-ad0140042890 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:01.601308 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 69/136] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:01 2022] GET /identity/v3/projects/f2e8bda5cd0745bbb7a1fd43ceeadc7a => generated 312 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:01.610822 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8cbf4-ed63-4e31-8d08-27f06ccae6ee None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:01.840556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8cbf4-ed63-4e31-8d08-27f06ccae6ee None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:01.894375 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5096851-d01e-4178-a18f-9f04a7931f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:01.895561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5096851-d01e-4178-a18f-9f04a7931f70 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:01.896047 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5096851-d01e-4178-a18f-9f04a7931f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:01.896522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5096851-d01e-4178-a18f-9f04a7931f70 None admin] PATH_INFO: `/v3/projects/f2e8bda5cd0745bbb7a1fd43ceeadc7a/users/93500b88e77e4d4ba8f347e33daa2e69/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:01.901755 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5096851-d01e-4178-a18f-9f04a7931f70 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2e8bda5cd0745bbb7a1fd43ceeadc7a, user_id=93500b88e77e4d4ba8f347e33daa2e69, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:01.903678 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5096851-d01e-4178-a18f-9f04a7931f70 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:01.936621 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 68/137] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:29:01 2022] PUT /identity/v3/projects/f2e8bda5cd0745bbb7a1fd43ceeadc7a/users/93500b88e77e4d4ba8f347e33daa2e69/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:03.136769 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bbc70-00e1-4cc4-ac72-c9c2043bddd8 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.137348 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bbc70-00e1-4cc4-ac72-c9c2043bddd8 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.137808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bbc70-00e1-4cc4-ac72-c9c2043bddd8 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.140407 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 70/138] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:03.148993 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d95cf5-f97b-4576-9b99-19d8d0646b4e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.150030 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d95cf5-f97b-4576-9b99-19d8d0646b4e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.150524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d95cf5-f97b-4576-9b99-19d8d0646b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.182450 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-89d95cf5-f97b-4576-9b99-19d8d0646b4e None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:03.184222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-89d95cf5-f97b-4576-9b99-19d8d0646b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:03.194287 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:03.297029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 69/139] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:03.730872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9cbc3-ac67-4cc4-9fc9-35f8661d08fe None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.731439 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9cbc3-ac67-4cc4-9fc9-35f8661d08fe None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.731793 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9cbc3-ac67-4cc4-9fc9-35f8661d08fe None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.733341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 71/140] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:03.761189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e151cce-e6cf-4257-a733-37a1bb03f37f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:03.761189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e151cce-e6cf-4257-a733-37a1bb03f37f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:03.761189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e151cce-e6cf-4257-a733-37a1bb03f37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:03.795234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7e151cce-e6cf-4257-a733-37a1bb03f37f None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:03.797739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7e151cce-e6cf-4257-a733-37a1bb03f37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:03.890299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9cbc3-ac67-4cc4-9fc9-35f8661d08fe None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:03.927447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 70/141] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:04.099501 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9cbc3-ac67-4cc4-9fc9-35f8661d08fe None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.150174 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b7794-876d-4f78-8fa9-2543ff2be33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.150850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b7794-876d-4f78-8fa9-2543ff2be33d None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.151118 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b7794-876d-4f78-8fa9-2543ff2be33d None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.151395 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b7794-876d-4f78-8fa9-2543ff2be33d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.152671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b7794-876d-4f78-8fa9-2543ff2be33d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.154274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b7794-876d-4f78-8fa9-2543ff2be33d None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.155977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 72/142] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:29:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:04.165238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e151cce-e6cf-4257-a733-37a1bb03f37f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.276881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:04.314346 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e151cce-e6cf-4257-a733-37a1bb03f37f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.352920 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b7794-876d-4f78-8fa9-2543ff2be33d None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.354770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae8773-e7d0-4c3a-987c-625a2457f7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.355448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae8773-e7d0-4c3a-987c-625a2457f7d0 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.355732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae8773-e7d0-4c3a-987c-625a2457f7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.356005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae8773-e7d0-4c3a-987c-625a2457f7d0 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.356717 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae8773-e7d0-4c3a-987c-625a2457f7d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.358348 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae8773-e7d0-4c3a-987c-625a2457f7d0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.392088 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 71/143] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:29:04 2022] POST /identity/v3/users => generated 283 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:04.454015 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:04.515096 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b7794-876d-4f78-8fa9-2543ff2be33d None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.574027 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324072a7-960a-4ed5-88ad-d31f8cfc74d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.574719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324072a7-960a-4ed5-88ad-d31f8cfc74d6 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.574993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324072a7-960a-4ed5-88ad-d31f8cfc74d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.575413 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324072a7-960a-4ed5-88ad-d31f8cfc74d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.592863 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324072a7-960a-4ed5-88ad-d31f8cfc74d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:04.598848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-324072a7-960a-4ed5-88ad-d31f8cfc74d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:04.598848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:04.598848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:04.598848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:04.598848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:04.598848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:04.598848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:04.598848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:04.598848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:04.598848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:04.598848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:04.598848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:04.598848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:04.598848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:04.598848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:04.598848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:04.599787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:04.599787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:04.599787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:04.599787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:04.599787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:04.599787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:04.599787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:04.599787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:04.599787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:04.599787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:04.599787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:04.599787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:04.599787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:04.599787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:04.599787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:04.599787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:04.600652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:04.600652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:04.600652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:04.600652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:04.600652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:04.600652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:04.600652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:04.600652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:04.600652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:04.600652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:04.600652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:04.600652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:04.600652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 73/144] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:04.608550 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae8773-e7d0-4c3a-987c-625a2457f7d0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.632770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ee3ef-0bf3-4efa-bf8b-3f2a7f00664c None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.633085 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ee3ef-0bf3-4efa-bf8b-3f2a7f00664c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.633378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ee3ef-0bf3-4efa-bf8b-3f2a7f00664c None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.635059 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 74/145] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:04.641381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686602d7-c080-44a4-9f54-e11a6297e32e None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.641697 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686602d7-c080-44a4-9f54-e11a6297e32e None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.641909 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686602d7-c080-44a4-9f54-e11a6297e32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.681565 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-686602d7-c080-44a4-9f54-e11a6297e32e None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:04.682802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-686602d7-c080-44a4-9f54-e11a6297e32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:04.823471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae8773-e7d0-4c3a-987c-625a2457f7d0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:04.853276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 75/146] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:29:04.878393 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:04.889313 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d2507-7beb-46d5-94d7-d8b4f93afd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:04.889942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d2507-7beb-46d5-94d7-d8b4f93afd74 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:04.890447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d2507-7beb-46d5-94d7-d8b4f93afd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:04.893521 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d2507-7beb-46d5-94d7-d8b4f93afd74 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:04.894233 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d2507-7beb-46d5-94d7-d8b4f93afd74 None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:04.894538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d2507-7beb-46d5-94d7-d8b4f93afd74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:04.895942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d2507-7beb-46d5-94d7-d8b4f93afd74 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:04.905805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 72/147] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:04.911598 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686602d7-c080-44a4-9f54-e11a6297e32e None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:04.966006 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:05.020165 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686602d7-c080-44a4-9f54-e11a6297e32e None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.059872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52e833-61f6-4aae-b9ce-20ebac485be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.060565 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52e833-61f6-4aae-b9ce-20ebac485be7 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.060844 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52e833-61f6-4aae-b9ce-20ebac485be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.061116 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52e833-61f6-4aae-b9ce-20ebac485be7 None admin] PATH_INFO: `/v3/users/93500b88e77e4d4ba8f347e33daa2e69` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.062308 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52e833-61f6-4aae-b9ce-20ebac485be7 None admin] RBAC: Authorizing `identity:get_user(user_id=93500b88e77e4d4ba8f347e33daa2e69)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.063823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52e833-61f6-4aae-b9ce-20ebac485be7 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.065519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 76/148] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:04 2022] GET /identity/v3/users/93500b88e77e4d4ba8f347e33daa2e69 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:05.071197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d2507-7beb-46d5-94d7-d8b4f93afd74 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.216961 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d2507-7beb-46d5-94d7-d8b4f93afd74 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.231563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4346f-ca59-41b9-a738-ce9e81372b07 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.231563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4346f-ca59-41b9-a738-ce9e81372b07 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.231563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4346f-ca59-41b9-a738-ce9e81372b07 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.231563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 77/149] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:29:05.240015 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c142870-c13a-4ad1-bf97-26b58dd110ec None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.240539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c142870-c13a-4ad1-bf97-26b58dd110ec None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.241021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c142870-c13a-4ad1-bf97-26b58dd110ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.278294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-1c142870-c13a-4ad1-bf97-26b58dd110ec None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:05.279533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-1c142870-c13a-4ad1-bf97-26b58dd110ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:05.296591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5137de4-22e2-4217-b7bf-080a8a60144c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.297814 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5137de4-22e2-4217-b7bf-080a8a60144c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.298286 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5137de4-22e2-4217-b7bf-080a8a60144c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.298756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5137de4-22e2-4217-b7bf-080a8a60144c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.392785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 78/150] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:05.393837 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:05.394803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561664d7-0dba-4ca0-a399-cd805783e41f None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.395114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561664d7-0dba-4ca0-a399-cd805783e41f None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.395957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561664d7-0dba-4ca0-a399-cd805783e41f None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.397865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 79/151] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:05.407541 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ad0e0-82c2-4d9a-9f9f-a06ba25e4ff6 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.407840 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ad0e0-82c2-4d9a-9f9f-a06ba25e4ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.408100 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ad0e0-82c2-4d9a-9f9f-a06ba25e4ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.422787 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5137de4-22e2-4217-b7bf-080a8a60144c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.424412 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5137de4-22e2-4217-b7bf-080a8a60144c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.434346 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-280ad0e0-82c2-4d9a-9f9f-a06ba25e4ff6 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:05.436061 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-280ad0e0-82c2-4d9a-9f9f-a06ba25e4ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:05.500973 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:05.537571 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 80/152] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:05.539679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ad0e0-82c2-4d9a-9f9f-a06ba25e4ff6 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.587074 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 73/153] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:05.593751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5137de4-22e2-4217-b7bf-080a8a60144c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.755843 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:05.775075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5137de4-22e2-4217-b7bf-080a8a60144c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.818793 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ad0e0-82c2-4d9a-9f9f-a06ba25e4ff6 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:05.844905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e17898-ce3f-4ea2-a5bc-d45b285aefde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.845960 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e17898-ce3f-4ea2-a5bc-d45b285aefde None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.846456 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e17898-ce3f-4ea2-a5bc-d45b285aefde None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.847093 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e17898-ce3f-4ea2-a5bc-d45b285aefde None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.849611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e17898-ce3f-4ea2-a5bc-d45b285aefde None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:05.852588 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e17898-ce3f-4ea2-a5bc-d45b285aefde None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:05.861417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 74/154] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:05 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:05.862814 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e17898-ce3f-4ea2-a5bc-d45b285aefde None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.886690 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8b0aa-af59-448f-8685-3eb531281197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:05.887366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8b0aa-af59-448f-8685-3eb531281197 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:05.887695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8b0aa-af59-448f-8685-3eb531281197 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:05.887972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8b0aa-af59-448f-8685-3eb531281197 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:05.916352 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8b0aa-af59-448f-8685-3eb531281197 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:05.921472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-a5d8b0aa-af59-448f-8685-3eb531281197 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:05.921472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:05.921472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:05.921472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:05.921472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:05.921472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:05.921472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:05.921472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:05.921472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:05.921472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:05.921472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:05.921472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:05.921472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:05.921472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:05.921472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:05.921472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:05.922533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:05.922533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:05.922533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:05.922533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:05.922533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:05.922533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:05.922533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:05.922533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:05.922533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:05.922533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:05.922533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:05.922533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:05.922533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:05.922533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:05.922533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:05.922533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:05.923322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:05.923322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:05.923322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:05.923322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:05.923322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:05.923322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:05.923322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:05.923322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:05.923322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:05.923322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:05.923322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:05.923322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:05.923322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 81/155] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:29:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:05.923966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8b0aa-af59-448f-8685-3eb531281197 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:05.994748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:06.096992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8b0aa-af59-448f-8685-3eb531281197 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.102634 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e17898-ce3f-4ea2-a5bc-d45b285aefde None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.157317 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3ef84-b60e-4164-b97d-ef2f8a464863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:06.158002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3ef84-b60e-4164-b97d-ef2f8a464863 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.158294 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3ef84-b60e-4164-b97d-ef2f8a464863 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.158551 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3ef84-b60e-4164-b97d-ef2f8a464863 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.166952 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb160dd-7f61-4ba3-9049-24c96e41060b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:06.168408 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb160dd-7f61-4ba3-9049-24c96e41060b None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.169317 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb160dd-7f61-4ba3-9049-24c96e41060b None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.170073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb160dd-7f61-4ba3-9049-24c96e41060b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.171510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb160dd-7f61-4ba3-9049-24c96e41060b None admin] RBAC: Adding query filter params (role.id=bb454faf5dd643b892c6f9b670ad1f2d, scope.project.id=4e48d801558a426b8b994cb2bab00a20, user.id=93500b88e77e4d4ba8f347e33daa2e69) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:06.171795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3ef84-b60e-4164-b97d-ef2f8a464863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:06.172627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb160dd-7f61-4ba3-9049-24c96e41060b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:06.175825 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb160dd-7f61-4ba3-9049-24c96e41060b None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:06.176734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b8d3ef84-b60e-4164-b97d-ef2f8a464863 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:29:06.176734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:06.176734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:06.176734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:06.176734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:06.176734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:06.176734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:06.176734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:06.176734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:06.176734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:06.176734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:06.176734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:06.176734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:06.176734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:06.176734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:06.176734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:06.178757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:06.178757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:06.178757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:06.178757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:06.178757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:06.178757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:06.178757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:06.178757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:06.178757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:06.178757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:06.178757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:06.178757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:06.178757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:06.178757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:06.178757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:06.178757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:06.180417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:06.180417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:06.180417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:06.180417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:06.180417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:06.180417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:06.180417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:06.180417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:06.180417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:06.180417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:06.180417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:29:06.180417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:06.180417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 75/156] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:29:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:06.180417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3ef84-b60e-4164-b97d-ef2f8a464863 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:06.199597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 82/157] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:29:05 2022] GET /identity/v3/role_assignments?user.id=93500b88e77e4d4ba8f347e33daa2e69&scope.project.id=4e48d801558a426b8b994cb2bab00a20&role.id=bb454faf5dd643b892c6f9b670ad1f2d => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:06.200625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb160dd-7f61-4ba3-9049-24c96e41060b None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:06.221075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:06.382002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb160dd-7f61-4ba3-9049-24c96e41060b None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.385163 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:06.428991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced1938-06d8-4c9b-aa6f-f3a3393547dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:06.430292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced1938-06d8-4c9b-aa6f-f3a3393547dd None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.430820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced1938-06d8-4c9b-aa6f-f3a3393547dd None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.431324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced1938-06d8-4c9b-aa6f-f3a3393547dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.445312 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced1938-06d8-4c9b-aa6f-f3a3393547dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:06.452081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-1ced1938-06d8-4c9b-aa6f-f3a3393547dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:29:06.452081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:06.452081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:06.452081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:06.452081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:06.452081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:06.452081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:06.452081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:06.452081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:06.452081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:06.452081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:06.452081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:06.452081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:06.452081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:06.452081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:06.452081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:06.453032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:06.453032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:06.453032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:06.453032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:06.453032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:06.453032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:06.453032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:06.453032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:06.453032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:06.453032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:06.453032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:06.453032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:06.453032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:06.453032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:06.453032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:06.453032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:06.453955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:06.453955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:06.453955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:06.453955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:06.453955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:06.453955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:06.453955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:06.453955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:06.453955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:06.453955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:06.453955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:29:06.453955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:06.461129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 83/158] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:06.462877 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced1938-06d8-4c9b-aa6f-f3a3393547dd None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:06.499241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3ef84-b60e-4164-b97d-ef2f8a464863 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.590904 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced1938-06d8-4c9b-aa6f-f3a3393547dd None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.631474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b138a6c-7459-4627-9a8a-7aa268e671fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:06.632178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b138a6c-7459-4627-9a8a-7aa268e671fd None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.632384 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b138a6c-7459-4627-9a8a-7aa268e671fd None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.632650 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b138a6c-7459-4627-9a8a-7aa268e671fd None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.633351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b138a6c-7459-4627-9a8a-7aa268e671fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:06.633658 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b138a6c-7459-4627-9a8a-7aa268e671fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:06.634157 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd18c80-85d7-4be1-bd9a-cbf74de0cbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:06.635089 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b138a6c-7459-4627-9a8a-7aa268e671fd None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:06.635197 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd18c80-85d7-4be1-bd9a-cbf74de0cbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.635655 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd18c80-85d7-4be1-bd9a-cbf74de0cbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.636153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd18c80-85d7-4be1-bd9a-cbf74de0cbf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.637232 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd18c80-85d7-4be1-bd9a-cbf74de0cbf6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:06.637799 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd18c80-85d7-4be1-bd9a-cbf74de0cbf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:06.639396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd18c80-85d7-4be1-bd9a-cbf74de0cbf6 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:06.649970 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 84/159] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:06.651608 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e56395-f683-4a81-9dc4-76012a07df57 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.652123 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e56395-f683-4a81-9dc4-76012a07df57 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.652387 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 76/160] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:29:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 474 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:06.652820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e56395-f683-4a81-9dc4-76012a07df57 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.653366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b138a6c-7459-4627-9a8a-7aa268e671fd None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:06.655215 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 85/161] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:06.662535 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9d6f9-1d25-4bea-8bee-107ef16b91ef None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.663386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9d6f9-1d25-4bea-8bee-107ef16b91ef None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.663881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9d6f9-1d25-4bea-8bee-107ef16b91ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.684567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-15c9d6f9-1d25-4bea-8bee-107ef16b91ef None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:06.686188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-15c9d6f9-1d25-4bea-8bee-107ef16b91ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:06.709560 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:06.764238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 86/162] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:29:06.765641 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9d6f9-1d25-4bea-8bee-107ef16b91ef None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:06.844383 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:06.882111 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9d6f9-1d25-4bea-8bee-107ef16b91ef None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.884041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b138a6c-7459-4627-9a8a-7aa268e671fd None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:06.920360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434f598-99bb-445a-9a11-7ccf44a0dca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:06.921681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434f598-99bb-445a-9a11-7ccf44a0dca8 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.922157 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434f598-99bb-445a-9a11-7ccf44a0dca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.922716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434f598-99bb-445a-9a11-7ccf44a0dca8 None admin] PATH_INFO: `/v3/users/93500b88e77e4d4ba8f347e33daa2e69` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.924165 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434f598-99bb-445a-9a11-7ccf44a0dca8 None admin] RBAC: Authorizing `identity:get_user(user_id=93500b88e77e4d4ba8f347e33daa2e69)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:06.925876 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434f598-99bb-445a-9a11-7ccf44a0dca8 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:06.927858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 87/163] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:06 2022] GET /identity/v3/users/93500b88e77e4d4ba8f347e33daa2e69 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:06.928826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434f598-99bb-445a-9a11-7ccf44a0dca8 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:06.957505 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:06.965939 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115d0c6-4bd1-43d7-9ed4-1341cb4ec7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:06.966580 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115d0c6-4bd1-43d7-9ed4-1341cb4ec7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:06.966902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115d0c6-4bd1-43d7-9ed4-1341cb4ec7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:06.967127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115d0c6-4bd1-43d7-9ed4-1341cb4ec7e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:06.967813 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115d0c6-4bd1-43d7-9ed4-1341cb4ec7e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:06.968155 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115d0c6-4bd1-43d7-9ed4-1341cb4ec7e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:06.969821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115d0c6-4bd1-43d7-9ed4-1341cb4ec7e2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:06.987889 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 77/164] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:06.988786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115d0c6-4bd1-43d7-9ed4-1341cb4ec7e2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:07.047998 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434f598-99bb-445a-9a11-7ccf44a0dca8 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:07.086027 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7039a1-102c-4881-b784-b27f8244d9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:07.087181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7039a1-102c-4881-b784-b27f8244d9dc None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:07.087716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7039a1-102c-4881-b784-b27f8244d9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:07.088298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7039a1-102c-4881-b784-b27f8244d9dc None admin] PATH_INFO: `/v3/users/7236c9ce10b9442581cf096a5b4cddf9` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:07.089750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7039a1-102c-4881-b784-b27f8244d9dc None admin] RBAC: Authorizing `identity:get_user(user_id=7236c9ce10b9442581cf096a5b4cddf9)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:07.091563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7039a1-102c-4881-b784-b27f8244d9dc None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:07.093644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 88/165] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:06 2022] GET /identity/v3/users/7236c9ce10b9442581cf096a5b4cddf9 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:07.094644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7039a1-102c-4881-b784-b27f8244d9dc None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:07.200041 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:07.206706 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7039a1-102c-4881-b784-b27f8244d9dc None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:07.249410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bc7e9-072a-41d6-b757-ce3b8b90a382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:07.250548 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bc7e9-072a-41d6-b757-ce3b8b90a382 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:07.251025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bc7e9-072a-41d6-b757-ce3b8b90a382 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:07.251495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bc7e9-072a-41d6-b757-ce3b8b90a382 None admin] PATH_INFO: `/v3/users/83569985bd5a40ccbdefde7c5a100678` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:07.253037 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121bc7e9-072a-41d6-b757-ce3b8b90a382 None admin] RBAC: Authorizing `identity:get_user(user_id=83569985bd5a40ccbdefde7c5a100678)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:07.254829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121bc7e9-072a-41d6-b757-ce3b8b90a382 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:07.256674 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 89/166] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:07 2022] GET /identity/v3/users/83569985bd5a40ccbdefde7c5a100678 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:07.262038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bc7e9-072a-41d6-b757-ce3b8b90a382 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:07.263544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115d0c6-4bd1-43d7-9ed4-1341cb4ec7e2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:07.329592 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909241c-4600-40e9-9cb6-fe8f516838de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:07.330239 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909241c-4600-40e9-9cb6-fe8f516838de None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:07.330540 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909241c-4600-40e9-9cb6-fe8f516838de None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:07.330915 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909241c-4600-40e9-9cb6-fe8f516838de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:07.347426 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:07.381282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bc7e9-072a-41d6-b757-ce3b8b90a382 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:07.421590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144cb3a-e72b-4d84-8ee1-e59f94c9f166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:07.423189 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144cb3a-e72b-4d84-8ee1-e59f94c9f166 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:07.423733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144cb3a-e72b-4d84-8ee1-e59f94c9f166 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:07.424220 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144cb3a-e72b-4d84-8ee1-e59f94c9f166 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:07.475940 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909241c-4600-40e9-9cb6-fe8f516838de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:07.477910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909241c-4600-40e9-9cb6-fe8f516838de None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:07.489539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:07.572943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144cb3a-e72b-4d84-8ee1-e59f94c9f166 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:07.575575 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144cb3a-e72b-4d84-8ee1-e59f94c9f166 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:07.654421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 78/167] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:07.655577 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909241c-4600-40e9-9cb6-fe8f516838de None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:07.738403 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 90/168] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:07.739655 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144cb3a-e72b-4d84-8ee1-e59f94c9f166 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:07.762665 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:07.803889 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:07.875888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144cb3a-e72b-4d84-8ee1-e59f94c9f166 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:07.887723 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909241c-4600-40e9-9cb6-fe8f516838de None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:07.939466 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6eaf3-5edc-45c1-bff5-e8bedceb3892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:07.940118 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6eaf3-5edc-45c1-bff5-e8bedceb3892 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:07.940398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6eaf3-5edc-45c1-bff5-e8bedceb3892 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:07.940670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6eaf3-5edc-45c1-bff5-e8bedceb3892 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:07.948473 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc246fb6-7d32-4c63-8562-a98b9b60945a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:07.949121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc246fb6-7d32-4c63-8562-a98b9b60945a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:07.949417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc246fb6-7d32-4c63-8562-a98b9b60945a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:07.949698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc246fb6-7d32-4c63-8562-a98b9b60945a None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:07.951070 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc246fb6-7d32-4c63-8562-a98b9b60945a None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:07.952545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc246fb6-7d32-4c63-8562-a98b9b60945a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:07.955384 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 79/169] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:07 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:07.956177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc246fb6-7d32-4c63-8562-a98b9b60945a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:07.963528 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6eaf3-5edc-45c1-bff5-e8bedceb3892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:07.968440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-d0b6eaf3-5edc-45c1-bff5-e8bedceb3892 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:07.968440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:07.968440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:07.968440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:07.968440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:07.968440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:07.968440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:07.968440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:07.968440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:07.968440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:07.968440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:07.968440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:07.968440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:07.968440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:07.968440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:07.968440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:07.969442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:07.969442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:07.969442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:07.969442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:07.969442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:07.969442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:07.969442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:07.969442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:07.969442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:07.969442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:07.969442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:07.969442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:07.969442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:07.969442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:07.969442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:07.969442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:07.970264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:07.970264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:07.970264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:07.970264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:07.970264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:07.970264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:07.970264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:07.970264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:07.970264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:07.970264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:07.970264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:07.970264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:07.970860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 91/170] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:07.976222 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6eaf3-5edc-45c1-bff5-e8bedceb3892 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:08.126638 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6eaf3-5edc-45c1-bff5-e8bedceb3892 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:08.154338 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:08.181229 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498072f1-0007-4616-bfd3-1a14fb3689b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:08.182262 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498072f1-0007-4616-bfd3-1a14fb3689b4 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:08.182731 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498072f1-0007-4616-bfd3-1a14fb3689b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:08.183190 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498072f1-0007-4616-bfd3-1a14fb3689b4 None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:08.184725 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498072f1-0007-4616-bfd3-1a14fb3689b4 None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:08.186493 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498072f1-0007-4616-bfd3-1a14fb3689b4 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:08.187427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc246fb6-7d32-4c63-8562-a98b9b60945a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:08.188997 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 92/171] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:07 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:08.190058 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498072f1-0007-4616-bfd3-1a14fb3689b4 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:08.221568 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:08.262016 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29ac3b-7ebb-427b-87fc-3444b88d04bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:08.262751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29ac3b-7ebb-427b-87fc-3444b88d04bf None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:08.263030 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29ac3b-7ebb-427b-87fc-3444b88d04bf None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:08.263301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29ac3b-7ebb-427b-87fc-3444b88d04bf None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:08.264620 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29ac3b-7ebb-427b-87fc-3444b88d04bf None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:08.266171 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29ac3b-7ebb-427b-87fc-3444b88d04bf None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:08.267953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 80/172] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:07 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:08.269104 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29ac3b-7ebb-427b-87fc-3444b88d04bf None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:08.305770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498072f1-0007-4616-bfd3-1a14fb3689b4 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:08.345632 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d16496-179f-418f-8aac-111ea798c66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:08.346730 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d16496-179f-418f-8aac-111ea798c66a None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:08.347196 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d16496-179f-418f-8aac-111ea798c66a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:08.347651 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d16496-179f-418f-8aac-111ea798c66a None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20/users/83569985bd5a40ccbdefde7c5a100678/roles/27a25b7936b8463c9e6d34d2cb49e61f` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:08.350129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d16496-179f-418f-8aac-111ea798c66a None admin] RBAC: Authorizing `identity:create_grant(project_id=4e48d801558a426b8b994cb2bab00a20, user_id=83569985bd5a40ccbdefde7c5a100678, role_id=27a25b7936b8463c9e6d34d2cb49e61f)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:08.351880 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d16496-179f-418f-8aac-111ea798c66a None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:08.373304 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 93/173] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:29:08 2022] PUT /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20/users/83569985bd5a40ccbdefde7c5a100678/roles/27a25b7936b8463c9e6d34d2cb49e61f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:08.374546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d16496-179f-418f-8aac-111ea798c66a None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:08.451141 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29ac3b-7ebb-427b-87fc-3444b88d04bf None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:08.475757 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:08.529630 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0948989-b829-411f-8114-ac19a77adab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:08.530576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0948989-b829-411f-8114-ac19a77adab5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:08.531207 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0948989-b829-411f-8114-ac19a77adab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:08.533096 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0948989-b829-411f-8114-ac19a77adab5 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:08.534026 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0948989-b829-411f-8114-ac19a77adab5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:08.534495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0948989-b829-411f-8114-ac19a77adab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:08.537396 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0948989-b829-411f-8114-ac19a77adab5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:08.558133 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 81/174] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:08.560504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0948989-b829-411f-8114-ac19a77adab5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:08.581562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d16496-179f-418f-8aac-111ea798c66a None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:08.628064 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d0908-30af-409d-a61d-102b8b149d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:08.628971 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d0908-30af-409d-a61d-102b8b149d42 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:08.629437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d0908-30af-409d-a61d-102b8b149d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:08.630103 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:08.631070 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d0908-30af-409d-a61d-102b8b149d42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:08.632109 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d0908-30af-409d-a61d-102b8b149d42 None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=4e48d801558a426b8b994cb2bab00a20, user.id=7236c9ce10b9442581cf096a5b4cddf9) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:08.632564 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d0908-30af-409d-a61d-102b8b149d42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:08.634208 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d0908-30af-409d-a61d-102b8b149d42 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:08.641007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 94/175] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:29:08 2022] GET /identity/v3/role_assignments?user.id=7236c9ce10b9442581cf096a5b4cddf9&scope.project.id=4e48d801558a426b8b994cb2bab00a20&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:08.642501 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8759c-5201-44a0-a194-8b6bed4cb7c2 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:08.642996 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8759c-5201-44a0-a194-8b6bed4cb7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:08.643441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8759c-5201-44a0-a194-8b6bed4cb7c2 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:08.644881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 95/176] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:08.645871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8759c-5201-44a0-a194-8b6bed4cb7c2 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:08.718534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0948989-b829-411f-8114-ac19a77adab5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:08.764911 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8759c-5201-44a0-a194-8b6bed4cb7c2 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:08.773334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2cca5-e690-4d9b-af52-eb3c4da24639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:08.774268 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2cca5-e690-4d9b-af52-eb3c4da24639 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:08.774717 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2cca5-e690-4d9b-af52-eb3c4da24639 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:08.775152 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2cca5-e690-4d9b-af52-eb3c4da24639 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:08.776075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2cca5-e690-4d9b-af52-eb3c4da24639 None admin] RBAC: Adding query filter params (role.id=05e78a4a009c41488cf3d2c47e8cbe30, scope.project.id=4e48d801558a426b8b994cb2bab00a20, user.id=93500b88e77e4d4ba8f347e33daa2e69) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:08.776557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2cca5-e690-4d9b-af52-eb3c4da24639 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:08.777163 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:08.779161 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2cca5-e690-4d9b-af52-eb3c4da24639 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:08.786364 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 82/177] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:29:08 2022] GET /identity/v3/role_assignments?user.id=93500b88e77e4d4ba8f347e33daa2e69&scope.project.id=4e48d801558a426b8b994cb2bab00a20&role.id=05e78a4a009c41488cf3d2c47e8cbe30 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:08.787940 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafa74b-acba-4091-ac6a-c262ac2fd169 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:08.788404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafa74b-acba-4091-ac6a-c262ac2fd169 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:08.788843 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafa74b-acba-4091-ac6a-c262ac2fd169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:08.806143 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394a152-25cf-4952-a87c-1d622d43d71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:08.806803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394a152-25cf-4952-a87c-1d622d43d71f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:08.807082 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394a152-25cf-4952-a87c-1d622d43d71f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:08.807355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394a152-25cf-4952-a87c-1d622d43d71f None admin] PATH_INFO: `/v3/users/93500b88e77e4d4ba8f347e33daa2e69` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:08.808533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394a152-25cf-4952-a87c-1d622d43d71f None admin] RBAC: Authorizing `identity:get_user(user_id=93500b88e77e4d4ba8f347e33daa2e69)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:08.810142 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394a152-25cf-4952-a87c-1d622d43d71f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:08.810848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-4dafa74b-acba-4091-ac6a-c262ac2fd169 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:08.811784 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 96/178] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:08 2022] GET /identity/v3/users/93500b88e77e4d4ba8f347e33daa2e69 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:08.812530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4dafa74b-acba-4091-ac6a-c262ac2fd169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:08.812954 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c899bf-1881-4380-b37c-eb54b11adec0 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:08.813277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c899bf-1881-4380-b37c-eb54b11adec0 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:08.813553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c899bf-1881-4380-b37c-eb54b11adec0 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:08.814986 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 97/179] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:08.819159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c899bf-1881-4380-b37c-eb54b11adec0 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:08.922016 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 83/180] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:08.924673 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44904fdc-5ca0-437e-b056-6de4fff7b3e7 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:08.925607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44904fdc-5ca0-437e-b056-6de4fff7b3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:08.927475 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44904fdc-5ca0-437e-b056-6de4fff7b3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:08.982548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-44904fdc-5ca0-437e-b056-6de4fff7b3e7 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:08.983710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-44904fdc-5ca0-437e-b056-6de4fff7b3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:09.016994 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c899bf-1881-4380-b37c-eb54b11adec0 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:09.075040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:09.085104 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a34bd-02ed-40f4-9eaa-3fb1f4d1ad86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:09.086032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a34bd-02ed-40f4-9eaa-3fb1f4d1ad86 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:09.086446 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a34bd-02ed-40f4-9eaa-3fb1f4d1ad86 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:09.086850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a34bd-02ed-40f4-9eaa-3fb1f4d1ad86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:09.152149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:09.177033 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a34bd-02ed-40f4-9eaa-3fb1f4d1ad86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:09.178699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a34bd-02ed-40f4-9eaa-3fb1f4d1ad86 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:09.194671 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 84/181] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:09.195835 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a584cd0-fe32-41d8-86be-96614bc515e8 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:09.196120 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a584cd0-fe32-41d8-86be-96614bc515e8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:09.196398 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a584cd0-fe32-41d8-86be-96614bc515e8 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:09.198262 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 85/182] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:09.211845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d3381-d0db-4e91-8fe3-d21d8eac5150 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:09.212149 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d3381-d0db-4e91-8fe3-d21d8eac5150 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:09.212392 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d3381-d0db-4e91-8fe3-d21d8eac5150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:09.241461 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-fe0d3381-d0db-4e91-8fe3-d21d8eac5150 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:09.242876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-fe0d3381-d0db-4e91-8fe3-d21d8eac5150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:09.320552 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 86/183] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:29:09.322858 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d3381-d0db-4e91-8fe3-d21d8eac5150 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:09.423050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 98/184] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:09.426587 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a34bd-02ed-40f4-9eaa-3fb1f4d1ad86 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:09.521245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:09.523191 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d3381-d0db-4e91-8fe3-d21d8eac5150 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:09.542182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:09.560708 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955f838-e20a-488a-a927-e17a6ee1fbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:09.561397 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955f838-e20a-488a-a927-e17a6ee1fbb5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:09.561677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955f838-e20a-488a-a927-e17a6ee1fbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:09.561949 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955f838-e20a-488a-a927-e17a6ee1fbb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:09.571219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a34bd-02ed-40f4-9eaa-3fb1f4d1ad86 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:09.573543 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955f838-e20a-488a-a927-e17a6ee1fbb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:09.578711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b955f838-e20a-488a-a927-e17a6ee1fbb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:29:09.578711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:09.578711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:09.578711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:09.578711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:09.578711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:09.578711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:09.578711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:09.578711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:09.578711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:09.578711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:09.578711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:09.578711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:09.578711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:09.578711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:09.578711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:09.579629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:09.579629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:09.579629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:09.579629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:09.579629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:09.579629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:09.579629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:09.579629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:09.579629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:09.579629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:09.579629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:09.579629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:09.579629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:09.579629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:09.579629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:09.579629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:09.580415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:09.580415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:09.580415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:09.580415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:09.580415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:09.580415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:09.580415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:09.580415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:09.580415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:09.580415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:09.580415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:29:09.580415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:09.580415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 87/185] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:29:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:09.581064 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955f838-e20a-488a-a927-e17a6ee1fbb5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:09.612393 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93cd7e-0611-47a0-bd76-72b3793b3682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:09.613038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93cd7e-0611-47a0-bd76-72b3793b3682 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:09.613350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93cd7e-0611-47a0-bd76-72b3793b3682 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:09.613626 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93cd7e-0611-47a0-bd76-72b3793b3682 None admin] PATH_INFO: `/v3/projects/f2e8bda5cd0745bbb7a1fd43ceeadc7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:09.614809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93cd7e-0611-47a0-bd76-72b3793b3682 None admin] RBAC: Authorizing `identity:get_project(project_id=f2e8bda5cd0745bbb7a1fd43ceeadc7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:09.616965 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93cd7e-0611-47a0-bd76-72b3793b3682 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:09.619025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 99/186] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:09 2022] GET /identity/v3/projects/f2e8bda5cd0745bbb7a1fd43ceeadc7a => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:09.619862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93cd7e-0611-47a0-bd76-72b3793b3682 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:09.712042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955f838-e20a-488a-a927-e17a6ee1fbb5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:09.748718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0a841-a728-47d2-99c1-ebfd7beb2bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:09.749382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0a841-a728-47d2-99c1-ebfd7beb2bcf None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:09.749658 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0a841-a728-47d2-99c1-ebfd7beb2bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:09.749920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0a841-a728-47d2-99c1-ebfd7beb2bcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:09.751032 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0a841-a728-47d2-99c1-ebfd7beb2bcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:09.752486 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0a841-a728-47d2-99c1-ebfd7beb2bcf None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:09.761095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 88/187] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:29:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:09.761095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0a841-a728-47d2-99c1-ebfd7beb2bcf None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:09.770731 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93cd7e-0611-47a0-bd76-72b3793b3682 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:09.781498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:09.810314 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1a519-29af-4dbd-9758-6f927f67f703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:09.810980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1a519-29af-4dbd-9758-6f927f67f703 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:09.811263 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1a519-29af-4dbd-9758-6f927f67f703 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:09.811535 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1a519-29af-4dbd-9758-6f927f67f703 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:09.812226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1a519-29af-4dbd-9758-6f927f67f703 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:09.812503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1a519-29af-4dbd-9758-6f927f67f703 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:09.813904 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1a519-29af-4dbd-9758-6f927f67f703 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:09.823109 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 100/188] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:09.823812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1a519-29af-4dbd-9758-6f927f67f703 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:09.835797 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:09.865448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0a841-a728-47d2-99c1-ebfd7beb2bcf None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:09.900580 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087be937-ad73-4be0-a971-b9d38d055f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:09.901219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087be937-ad73-4be0-a971-b9d38d055f61 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:09.901494 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087be937-ad73-4be0-a971-b9d38d055f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:09.901767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087be937-ad73-4be0-a971-b9d38d055f61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:09.902485 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087be937-ad73-4be0-a971-b9d38d055f61 None admin] RBAC: Adding query filter params (role.id=bb454faf5dd643b892c6f9b670ad1f2d, scope.project.id=f2e8bda5cd0745bbb7a1fd43ceeadc7a, user.id=93500b88e77e4d4ba8f347e33daa2e69) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:09.902788 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087be937-ad73-4be0-a971-b9d38d055f61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:09.904576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087be937-ad73-4be0-a971-b9d38d055f61 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:09.910496 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 89/189] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:29:09 2022] GET /identity/v3/role_assignments?user.id=93500b88e77e4d4ba8f347e33daa2e69&scope.project.id=f2e8bda5cd0745bbb7a1fd43ceeadc7a&role.id=bb454faf5dd643b892c6f9b670ad1f2d => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:09.911183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087be937-ad73-4be0-a971-b9d38d055f61 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:09.951370 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1a519-29af-4dbd-9758-6f927f67f703 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:09.998821 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71fb6a-9108-4799-b5a7-5118364e0991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:10.000199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71fb6a-9108-4799-b5a7-5118364e0991 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:10.000714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71fb6a-9108-4799-b5a7-5118364e0991 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:10.001215 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71fb6a-9108-4799-b5a7-5118364e0991 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:10.019910 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71fb6a-9108-4799-b5a7-5118364e0991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:10.027544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-5e71fb6a-9108-4799-b5a7-5118364e0991 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:10.027544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:10.027544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:10.027544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:10.027544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:10.027544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:10.027544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:10.027544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:10.027544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:10.027544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:10.027544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:10.027544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:10.027544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:10.027544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:10.027544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:10.027544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:10.028567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:10.028567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:10.028567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:10.028567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:10.028567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:10.028567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:10.028567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:10.028567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:10.028567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:10.028567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:10.028567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:10.028567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:10.028567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:10.028567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:10.028567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:10.028567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:10.029389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:10.029389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:10.029389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:10.029389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:10.029389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:10.029389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:10.029389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:10.029389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:10.029389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:10.029389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:10.029389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:10.029389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:10.032088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 101/190] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:29:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:10.033128 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71fb6a-9108-4799-b5a7-5118364e0991 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:10.046153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:10.076132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087be937-ad73-4be0-a971-b9d38d055f61 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:10.128248 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:10.132374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881724b4-5452-4104-8fb2-d4f289d81a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:10.133015 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881724b4-5452-4104-8fb2-d4f289d81a3d None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:10.133352 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881724b4-5452-4104-8fb2-d4f289d81a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:10.133640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881724b4-5452-4104-8fb2-d4f289d81a3d None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:10.134647 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881724b4-5452-4104-8fb2-d4f289d81a3d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:10.136102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881724b4-5452-4104-8fb2-d4f289d81a3d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:10.165774 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 90/191] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:29:09 2022] POST /identity/v3/users => generated 297 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:10.171368 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881724b4-5452-4104-8fb2-d4f289d81a3d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:10.181767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71fb6a-9108-4799-b5a7-5118364e0991 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:10.255994 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2717e6d-6852-475f-9a18-2f6f72140b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:10.257721 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2717e6d-6852-475f-9a18-2f6f72140b2e None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:10.258502 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2717e6d-6852-475f-9a18-2f6f72140b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:10.259269 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2717e6d-6852-475f-9a18-2f6f72140b2e None admin] PATH_INFO: `/v3/users/499e4611aa4b4699aaa619a5f072bd87` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:10.290782 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2717e6d-6852-475f-9a18-2f6f72140b2e None admin] RBAC: Authorizing `identity:get_user(user_id=499e4611aa4b4699aaa619a5f072bd87)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:10.296608 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2717e6d-6852-475f-9a18-2f6f72140b2e None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:10.300936 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 102/192] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:10 2022] GET /identity/v3/users/499e4611aa4b4699aaa619a5f072bd87 => generated 283 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:10.317010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2717e6d-6852-475f-9a18-2f6f72140b2e None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:10.332242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881724b4-5452-4104-8fb2-d4f289d81a3d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:10.415366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91f29b-2231-4a75-abe7-beaaada61f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:10.416012 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91f29b-2231-4a75-abe7-beaaada61f5a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:10.416610 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91f29b-2231-4a75-abe7-beaaada61f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:10.416901 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91f29b-2231-4a75-abe7-beaaada61f5a None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:10.417755 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91f29b-2231-4a75-abe7-beaaada61f5a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:10.418065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91f29b-2231-4a75-abe7-beaaada61f5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:10.421164 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91f29b-2231-4a75-abe7-beaaada61f5a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:10.435054 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 91/193] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:29:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:10.442021 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91f29b-2231-4a75-abe7-beaaada61f5a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:10.447422 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:10.491263 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2717e6d-6852-475f-9a18-2f6f72140b2e None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:10.543780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:10.557523 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122996a9-3e56-4fcf-90d0-69c5b0bb132c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:10.558606 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122996a9-3e56-4fcf-90d0-69c5b0bb132c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:10.559131 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122996a9-3e56-4fcf-90d0-69c5b0bb132c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:10.559701 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122996a9-3e56-4fcf-90d0-69c5b0bb132c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:10.609692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91f29b-2231-4a75-abe7-beaaada61f5a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:10.669293 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd938e25-8ae9-441f-802e-301f017acd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:10.669886 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd938e25-8ae9-441f-802e-301f017acd96 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:10.670165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd938e25-8ae9-441f-802e-301f017acd96 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:10.670440 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd938e25-8ae9-441f-802e-301f017acd96 None admin] PATH_INFO: `/v3/users/499e4611aa4b4699aaa619a5f072bd87` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:10.672611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd938e25-8ae9-441f-802e-301f017acd96 None admin] RBAC: Authorizing `identity:get_user(user_id=499e4611aa4b4699aaa619a5f072bd87)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:10.674284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd938e25-8ae9-441f-802e-301f017acd96 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:10.677739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 92/194] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:10 2022] GET /identity/v3/users/499e4611aa4b4699aaa619a5f072bd87 => generated 283 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:10.681862 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd938e25-8ae9-441f-802e-301f017acd96 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:10.694086 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122996a9-3e56-4fcf-90d0-69c5b0bb132c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:10.695673 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122996a9-3e56-4fcf-90d0-69c5b0bb132c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:10.793742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:10.821951 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd938e25-8ae9-441f-802e-301f017acd96 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:10.896006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08cd4d-9eba-478e-b17b-318b04cae232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:10.897172 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08cd4d-9eba-478e-b17b-318b04cae232 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:10.897678 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08cd4d-9eba-478e-b17b-318b04cae232 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:10.898120 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08cd4d-9eba-478e-b17b-318b04cae232 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:10.926604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 103/195] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:10.932192 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122996a9-3e56-4fcf-90d0-69c5b0bb132c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:10.972363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:11.027822 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08cd4d-9eba-478e-b17b-318b04cae232 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:11.032627 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08cd4d-9eba-478e-b17b-318b04cae232 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:11.115737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122996a9-3e56-4fcf-90d0-69c5b0bb132c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:11.139810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:11.159774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b1bdd-9c47-4648-946a-bf5723567575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:11.160422 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b1bdd-9c47-4648-946a-bf5723567575 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:11.160693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b1bdd-9c47-4648-946a-bf5723567575 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:11.160957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b1bdd-9c47-4648-946a-bf5723567575 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:11.162273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b1bdd-9c47-4648-946a-bf5723567575 None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:11.163803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b1bdd-9c47-4648-946a-bf5723567575 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:11.166063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 104/196] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:10 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:11.167242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e447aa-f8fa-4131-b3dc-4010a8a771c5 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:11.167611 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e447aa-f8fa-4131-b3dc-4010a8a771c5 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:11.167885 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e447aa-f8fa-4131-b3dc-4010a8a771c5 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:11.169238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 105/197] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:11.174488 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e447aa-f8fa-4131-b3dc-4010a8a771c5 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:11.175290 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 93/198] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 05:29:11.176867 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316c83d-7075-4e2b-b351-58575bf09084 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:11.177166 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316c83d-7075-4e2b-b351-58575bf09084 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:11.177462 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316c83d-7075-4e2b-b351-58575bf09084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:11.196368 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-9316c83d-7075-4e2b-b351-58575bf09084 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:11.197608 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9316c83d-7075-4e2b-b351-58575bf09084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:11.275534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 94/199] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:11.277027 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316c83d-7075-4e2b-b351-58575bf09084 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:11.310940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:11.330032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e447aa-f8fa-4131-b3dc-4010a8a771c5 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:11.398644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7680e-d3ae-4d8d-93eb-684f7437c6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:11.399567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7680e-d3ae-4d8d-93eb-684f7437c6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:11.400026 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7680e-d3ae-4d8d-93eb-684f7437c6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:11.400438 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7680e-d3ae-4d8d-93eb-684f7437c6b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:11.401356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7680e-d3ae-4d8d-93eb-684f7437c6b0 None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=b13122687f1640669b462d728fc006ce, user.id=499e4611aa4b4699aaa619a5f072bd87) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:11.401965 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7680e-d3ae-4d8d-93eb-684f7437c6b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:11.403586 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7680e-d3ae-4d8d-93eb-684f7437c6b0 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:11.410492 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 106/200] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:29:11 2022] GET /identity/v3/role_assignments?user.id=499e4611aa4b4699aaa619a5f072bd87&scope.project.id=b13122687f1640669b462d728fc006ce&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:11.438300 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316c83d-7075-4e2b-b351-58575bf09084 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:11.439925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:11.493222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaf31f-141f-47fe-b54f-d9f78f248c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:11.493908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaf31f-141f-47fe-b54f-d9f78f248c62 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:11.494186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaf31f-141f-47fe-b54f-d9f78f248c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:11.494458 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaf31f-141f-47fe-b54f-d9f78f248c62 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:11.495798 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaf31f-141f-47fe-b54f-d9f78f248c62 None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:11.497451 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaf31f-141f-47fe-b54f-d9f78f248c62 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:11.499253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 95/201] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:11 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:11.505393 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7680e-d3ae-4d8d-93eb-684f7437c6b0 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:11.615054 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaf31f-141f-47fe-b54f-d9f78f248c62 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:11.694597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7680e-d3ae-4d8d-93eb-684f7437c6b0 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:11.749305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372dcd1-bc74-4b51-a30b-8cd24085233b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:11.750634 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372dcd1-bc74-4b51-a30b-8cd24085233b None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:11.751393 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372dcd1-bc74-4b51-a30b-8cd24085233b None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:11.751856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372dcd1-bc74-4b51-a30b-8cd24085233b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:11.753184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372dcd1-bc74-4b51-a30b-8cd24085233b None admin] RBAC: Adding query filter params (role.id=05e78a4a009c41488cf3d2c47e8cbe30, scope.project.id=b13122687f1640669b462d728fc006ce, user.id=499e4611aa4b4699aaa619a5f072bd87) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:11.753717 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372dcd1-bc74-4b51-a30b-8cd24085233b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:11.756273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372dcd1-bc74-4b51-a30b-8cd24085233b None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:11.774588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 107/202] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:29:11 2022] GET /identity/v3/role_assignments?user.id=499e4611aa4b4699aaa619a5f072bd87&scope.project.id=b13122687f1640669b462d728fc006ce&role.id=05e78a4a009c41488cf3d2c47e8cbe30 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:11.799577 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaf31f-141f-47fe-b54f-d9f78f248c62 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:11.862620 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7580f-4905-4787-90b8-611a6366f6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:11.863858 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7580f-4905-4787-90b8-611a6366f6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:11.864374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7580f-4905-4787-90b8-611a6366f6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:11.865076 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7580f-4905-4787-90b8-611a6366f6c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:11.898380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7580f-4905-4787-90b8-611a6366f6c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:11.908234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-4cd7580f-4905-4787-90b8-611a6366f6c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:29:11.908234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:11.908234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:11.908234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:11.908234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:11.908234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:11.908234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:11.908234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:11.908234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:11.908234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:11.908234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:11.908234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:11.908234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:11.908234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:11.908234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:11.908234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:11.910071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:11.910071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:11.910071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:11.910071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:11.910071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:11.910071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:11.910071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:11.910071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:11.910071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:11.910071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:11.910071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:11.910071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:11.910071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:11.910071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:11.910071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:11.910071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:11.911489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:11.911489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:11.911489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:11.911489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:11.911489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:11.911489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:11.911489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:11.911489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:11.911489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:11.911489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:11.911489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:29:11.911489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:11.915776 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 96/203] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:11.923088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372dcd1-bc74-4b51-a30b-8cd24085233b None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:11.973643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:12.088497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372dcd1-bc74-4b51-a30b-8cd24085233b None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:12.132924 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1904e-8373-486e-92e3-5506aeef5bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:12.133606 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1904e-8373-486e-92e3-5506aeef5bdc None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:12.133881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1904e-8373-486e-92e3-5506aeef5bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:12.134165 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1904e-8373-486e-92e3-5506aeef5bdc None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:12.134849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1904e-8373-486e-92e3-5506aeef5bdc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:12.135151 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1904e-8373-486e-92e3-5506aeef5bdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:12.136571 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1904e-8373-486e-92e3-5506aeef5bdc None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:12.151139 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 108/204] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:12.153326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7580f-4905-4787-90b8-611a6366f6c8 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:12.216018 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:12.313011 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7580f-4905-4787-90b8-611a6366f6c8 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:12.353601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250693af-94fe-46f5-8322-43e4647b53de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:12.354254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250693af-94fe-46f5-8322-43e4647b53de None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:12.354532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250693af-94fe-46f5-8322-43e4647b53de None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:12.354806 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250693af-94fe-46f5-8322-43e4647b53de None admin] PATH_INFO: `/v3/users/83569985bd5a40ccbdefde7c5a100678` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:12.355988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250693af-94fe-46f5-8322-43e4647b53de None admin] RBAC: Authorizing `identity:get_user(user_id=83569985bd5a40ccbdefde7c5a100678)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:12.357766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250693af-94fe-46f5-8322-43e4647b53de None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:12.359729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 97/205] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:12 2022] GET /identity/v3/users/83569985bd5a40ccbdefde7c5a100678 => generated 289 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:12.365441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1904e-8373-486e-92e3-5506aeef5bdc None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:12.459960 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:12.478294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1904e-8373-486e-92e3-5506aeef5bdc None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:12.515197 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ed232-9d13-492d-ac3c-a887478eb4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:12.515728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ed232-9d13-492d-ac3c-a887478eb4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:12.516005 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ed232-9d13-492d-ac3c-a887478eb4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:12.516279 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ed232-9d13-492d-ac3c-a887478eb4b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:12.589400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ed232-9d13-492d-ac3c-a887478eb4b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:12.591117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ed232-9d13-492d-ac3c-a887478eb4b7 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:12.689787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:12.700646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 109/206] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:12.706984 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250693af-94fe-46f5-8322-43e4647b53de None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:12.816806 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:12.825515 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250693af-94fe-46f5-8322-43e4647b53de None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:12.874312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ff3cc-ddd5-4fcb-a29f-8622af38df95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:12.875029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ff3cc-ddd5-4fcb-a29f-8622af38df95 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:12.875313 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ff3cc-ddd5-4fcb-a29f-8622af38df95 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:12.875585 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ff3cc-ddd5-4fcb-a29f-8622af38df95 None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:12.876772 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ff3cc-ddd5-4fcb-a29f-8622af38df95 None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:12.878273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ff3cc-ddd5-4fcb-a29f-8622af38df95 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:12.884944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 98/207] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:12 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:12.886402 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ed232-9d13-492d-ac3c-a887478eb4b7 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:12.997669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ed232-9d13-492d-ac3c-a887478eb4b7 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:13.034300 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141d98b-1f4d-4c60-9d46-8ef5c2a0e4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:13.034924 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141d98b-1f4d-4c60-9d46-8ef5c2a0e4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:13.035190 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141d98b-1f4d-4c60-9d46-8ef5c2a0e4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:13.035464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141d98b-1f4d-4c60-9d46-8ef5c2a0e4c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:13.036205 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141d98b-1f4d-4c60-9d46-8ef5c2a0e4c1 None admin] RBAC: Adding query filter params (role.id=27a25b7936b8463c9e6d34d2cb49e61f, scope.project.id=4e48d801558a426b8b994cb2bab00a20, user.id=83569985bd5a40ccbdefde7c5a100678) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:13.036510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141d98b-1f4d-4c60-9d46-8ef5c2a0e4c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:13.037991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141d98b-1f4d-4c60-9d46-8ef5c2a0e4c1 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:13.047945 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 110/208] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:29:12 2022] GET /identity/v3/role_assignments?user.id=83569985bd5a40ccbdefde7c5a100678&scope.project.id=4e48d801558a426b8b994cb2bab00a20&role.id=27a25b7936b8463c9e6d34d2cb49e61f => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:13.084504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b800ad9-3307-4461-817e-8ee8b7ab9f07 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:13.084877 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b800ad9-3307-4461-817e-8ee8b7ab9f07 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:13.085099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b800ad9-3307-4461-817e-8ee8b7ab9f07 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:13.086877 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 99/209] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:13.092610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6f9fd-35ef-4c9e-99a6-8a772394d116 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:13.092994 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6f9fd-35ef-4c9e-99a6-8a772394d116 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:13.093288 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6f9fd-35ef-4c9e-99a6-8a772394d116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:13.112813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-aba6f9fd-35ef-4c9e-99a6-8a772394d116 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:13.118525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-aba6f9fd-35ef-4c9e-99a6-8a772394d116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:13.179072 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:13.208297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 111/210] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:13.599279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b800ad9-3307-4461-817e-8ee8b7ab9f07 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:13.705622 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b800ad9-3307-4461-817e-8ee8b7ab9f07 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:13.754500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362dcff-b2cf-4ac1-b86d-6aa0c038a208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:13.755748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362dcff-b2cf-4ac1-b86d-6aa0c038a208 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:13.755748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362dcff-b2cf-4ac1-b86d-6aa0c038a208 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:13.755748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362dcff-b2cf-4ac1-b86d-6aa0c038a208 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:13.782226 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362dcff-b2cf-4ac1-b86d-6aa0c038a208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:13.792948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-c362dcff-b2cf-4ac1-b86d-6aa0c038a208 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:13.792948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:13.792948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:13.792948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:13.792948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:13.792948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:13.792948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:13.792948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:13.792948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:13.792948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:13.792948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:13.792948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:13.792948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:13.792948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:13.792948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:13.792948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:13.794875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:13.794875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:13.794875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:13.794875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:13.794875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:13.794875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:13.794875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:13.794875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:13.794875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:13.794875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:13.794875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:13.794875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:13.794875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:13.794875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:13.794875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:13.794875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:13.796502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:13.796502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:13.796502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:13.796502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:13.796502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:13.796502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:13.796502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:13.796502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:13.796502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:13.796502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:13.796502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:13.796502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:13.800879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 100/211] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:13.810389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6f9fd-35ef-4c9e-99a6-8a772394d116 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:13.954641 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6f9fd-35ef-4c9e-99a6-8a772394d116 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:13.992898 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2800fb-ad0c-44e4-af72-67d5e07bac5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:13.993543 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2800fb-ad0c-44e4-af72-67d5e07bac5c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:13.993823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2800fb-ad0c-44e4-af72-67d5e07bac5c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:13.994066 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2800fb-ad0c-44e4-af72-67d5e07bac5c None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:13.994743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2800fb-ad0c-44e4-af72-67d5e07bac5c None admin] RBAC: Adding query filter params (name=member) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:13.995036 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2800fb-ad0c-44e4-af72-67d5e07bac5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:13.996417 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2800fb-ad0c-44e4-af72-67d5e07bac5c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:14.005984 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 112/212] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:14.011751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362dcff-b2cf-4ac1-b86d-6aa0c038a208 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:14.040459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:14.118788 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba1710-b9a1-4d24-a254-b654e18a3d43 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:14.119086 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba1710-b9a1-4d24-a254-b654e18a3d43 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:14.119436 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba1710-b9a1-4d24-a254-b654e18a3d43 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:14.120839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 113/213] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:29:14.126656 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed52d8-805a-411d-94e2-a23567e9a5fc None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:14.126956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed52d8-805a-411d-94e2-a23567e9a5fc None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:14.127408 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed52d8-805a-411d-94e2-a23567e9a5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:14.130920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362dcff-b2cf-4ac1-b86d-6aa0c038a208 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:14.146803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-aaed52d8-805a-411d-94e2-a23567e9a5fc None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:14.148160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-aaed52d8-805a-411d-94e2-a23567e9a5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:14.204170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3506bb-04f7-445b-b16d-fce8b68331e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:14.206472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3506bb-04f7-445b-b16d-fce8b68331e0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:14.206903 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3506bb-04f7-445b-b16d-fce8b68331e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:14.207747 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3506bb-04f7-445b-b16d-fce8b68331e0 None admin] PATH_INFO: `/v3/users/5f1ee136fc19419f9b4c3636e2125803` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:14.213588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:14.243273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3506bb-04f7-445b-b16d-fce8b68331e0 None admin] RBAC: Authorizing `identity:get_user(user_id=5f1ee136fc19419f9b4c3636e2125803)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:14.243733 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3506bb-04f7-445b-b16d-fce8b68331e0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:14.244305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 114/214] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:14.247318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 101/215] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:14 2022] GET /identity/v3/users/5f1ee136fc19419f9b4c3636e2125803 => generated 297 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:14.253024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed52d8-805a-411d-94e2-a23567e9a5fc None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:14.437319 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed52d8-805a-411d-94e2-a23567e9a5fc None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:14.502209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8ada6-e26c-4694-80a7-32b8570b74de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:14.503183 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8ada6-e26c-4694-80a7-32b8570b74de None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:14.503679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8ada6-e26c-4694-80a7-32b8570b74de None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:14.504186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8ada6-e26c-4694-80a7-32b8570b74de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:14.516757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3506bb-04f7-445b-b16d-fce8b68331e0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:14.573837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:14.612530 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8ada6-e26c-4694-80a7-32b8570b74de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:14.614469 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8ada6-e26c-4694-80a7-32b8570b74de None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:14.698674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:14.743135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3506bb-04f7-445b-b16d-fce8b68331e0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:14.746080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 115/216] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:14.751995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8ada6-e26c-4694-80a7-32b8570b74de None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:14.809843 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527b61d-e0a6-464c-b428-be56d1f9d906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:14.811879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527b61d-e0a6-464c-b428-be56d1f9d906 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:14.812530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527b61d-e0a6-464c-b428-be56d1f9d906 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:14.813177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527b61d-e0a6-464c-b428-be56d1f9d906 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:14.815421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7527b61d-e0a6-464c-b428-be56d1f9d906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:14.819369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7527b61d-e0a6-464c-b428-be56d1f9d906 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:14.822329 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 102/217] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:29:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:14.832002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527b61d-e0a6-464c-b428-be56d1f9d906 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:14.887475 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:14.949635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8ada6-e26c-4694-80a7-32b8570b74de None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:15.008643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b5fc3-97f4-4d83-ac5c-10a07331afcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:15.010049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b5fc3-97f4-4d83-ac5c-10a07331afcc None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:15.010705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b5fc3-97f4-4d83-ac5c-10a07331afcc None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:15.011335 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b5fc3-97f4-4d83-ac5c-10a07331afcc None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:15.013823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b5fc3-97f4-4d83-ac5c-10a07331afcc None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:15.016901 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b5fc3-97f4-4d83-ac5c-10a07331afcc None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:15.020727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 116/218] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:14 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:15.030958 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b5fc3-97f4-4d83-ac5c-10a07331afcc None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:15.046082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7527b61d-e0a6-464c-b428-be56d1f9d906 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:15.096875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb8287-0918-418c-bb3e-9ff6e940be31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:15.098187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb8287-0918-418c-bb3e-9ff6e940be31 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:15.098668 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb8287-0918-418c-bb3e-9ff6e940be31 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:15.099214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb8287-0918-418c-bb3e-9ff6e940be31 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:15.100456 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb8287-0918-418c-bb3e-9ff6e940be31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:15.103237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb8287-0918-418c-bb3e-9ff6e940be31 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:15.141122 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 103/219] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:29:14 2022] POST /identity/v3/users => generated 297 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:15.143409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca3d08-f430-452f-8967-7084927bb08d None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:15.143899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca3d08-f430-452f-8967-7084927bb08d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:15.144392 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca3d08-f430-452f-8967-7084927bb08d None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:15.154308 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 104/220] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:15 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:15.156701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb8f95-c096-41cd-b1a2-5622573f4a0c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:15.157209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb8f95-c096-41cd-b1a2-5622573f4a0c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:15.157688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb8f95-c096-41cd-b1a2-5622573f4a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:15.191588 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-dfcb8f95-c096-41cd-b1a2-5622573f4a0c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:15.193875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-dfcb8f95-c096-41cd-b1a2-5622573f4a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:15.197701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:15.202730 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b5fc3-97f4-4d83-ac5c-10a07331afcc None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:15.274966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9ef4a-0d8d-4453-97cd-c32062245bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:15.276080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9ef4a-0d8d-4453-97cd-c32062245bdd None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:15.276546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9ef4a-0d8d-4453-97cd-c32062245bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:15.277045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9ef4a-0d8d-4453-97cd-c32062245bdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:15.278668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9ef4a-0d8d-4453-97cd-c32062245bdd None admin] RBAC: Adding query filter params (role.id=bb454faf5dd643b892c6f9b670ad1f2d, scope.project.id=b13122687f1640669b462d728fc006ce, user.id=5f1ee136fc19419f9b4c3636e2125803) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:15.279132 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9ef4a-0d8d-4453-97cd-c32062245bdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:15.284540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9ef4a-0d8d-4453-97cd-c32062245bdd None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:15.300444 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 117/221] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:29:15 2022] GET /identity/v3/role_assignments?user.id=5f1ee136fc19419f9b4c3636e2125803&scope.project.id=b13122687f1640669b462d728fc006ce&role.id=bb454faf5dd643b892c6f9b670ad1f2d => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:15.331283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 105/222] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:15.406151 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89891a5-f4f1-40e1-abd1-fc82d4345fe6 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:15.406397 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89891a5-f4f1-40e1-abd1-fc82d4345fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:15.406641 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89891a5-f4f1-40e1-abd1-fc82d4345fe6 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:15.408030 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 118/223] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:15.415178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e014c-a4e1-4517-9461-e3f8b6439f03 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:15.416167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e014c-a4e1-4517-9461-e3f8b6439f03 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:15.417064 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e014c-a4e1-4517-9461-e3f8b6439f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:15.450025 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-922e014c-a4e1-4517-9461-e3f8b6439f03 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:15.452589 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-922e014c-a4e1-4517-9461-e3f8b6439f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:15.587946 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 106/224] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:15.699471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89891a5-f4f1-40e1-abd1-fc82d4345fe6 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:15.703616 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:15.891596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e014c-a4e1-4517-9461-e3f8b6439f03 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:15.913171 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89891a5-f4f1-40e1-abd1-fc82d4345fe6 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:15.985865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9026c9-9e1a-474c-97d5-15fc3b3dfcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:15.986539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9026c9-9e1a-474c-97d5-15fc3b3dfcbc None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:15.986816 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9026c9-9e1a-474c-97d5-15fc3b3dfcbc None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:15.987427 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9026c9-9e1a-474c-97d5-15fc3b3dfcbc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:15.999637 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9026c9-9e1a-474c-97d5-15fc3b3dfcbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:16.005032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-0b9026c9-9e1a-474c-97d5-15fc3b3dfcbc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:16.005032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:16.005032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:16.005032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:16.005032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:16.005032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:16.005032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:16.005032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:16.005032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:16.005032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:16.005032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:16.005032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:16.005032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:16.005032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:16.005032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:16.005032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:16.006736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:16.006736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:16.006736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:16.006736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:16.006736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:16.006736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:16.006736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:16.006736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:16.006736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:16.006736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:16.006736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:16.006736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:16.006736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:16.006736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:16.006736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:16.006736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:16.008277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:16.008277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:16.008277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:16.008277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:16.008277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:16.008277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:16.008277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:16.008277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:16.008277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:16.008277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:16.008277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:16.008277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:16.008277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 119/225] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:29:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:16.015723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9026c9-9e1a-474c-97d5-15fc3b3dfcbc None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:16.070307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:16.097065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:16.130303 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9026c9-9e1a-474c-97d5-15fc3b3dfcbc None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:16.138720 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e014c-a4e1-4517-9461-e3f8b6439f03 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:16.167425 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aba33a-9922-4a70-8394-81639960ea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:16.168210 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aba33a-9922-4a70-8394-81639960ea1b None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:16.168671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aba33a-9922-4a70-8394-81639960ea1b None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:16.169070 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aba33a-9922-4a70-8394-81639960ea1b None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:16.169953 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aba33a-9922-4a70-8394-81639960ea1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:16.170496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aba33a-9922-4a70-8394-81639960ea1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:16.172078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aba33a-9922-4a70-8394-81639960ea1b None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:16.181697 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 120/226] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:29:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:16.187609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aba33a-9922-4a70-8394-81639960ea1b None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:16.216646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179e3e6-b608-44f8-b87d-e09c2823dc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:16.217888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179e3e6-b608-44f8-b87d-e09c2823dc8c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:16.218380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179e3e6-b608-44f8-b87d-e09c2823dc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:16.219190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179e3e6-b608-44f8-b87d-e09c2823dc8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:16.262064 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179e3e6-b608-44f8-b87d-e09c2823dc8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:16.267135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-0179e3e6-b608-44f8-b87d-e09c2823dc8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:29:16.267135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:16.267135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:16.267135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:16.267135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:16.267135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:16.267135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:16.267135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:16.267135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:16.267135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:16.267135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:16.267135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:16.267135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:16.267135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:16.267135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:16.267135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:16.268049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:16.268049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:16.268049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:16.268049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:16.268049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:16.268049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:16.268049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:16.268049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:16.268049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:16.268049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:16.268049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:16.268049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:16.268049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:16.268049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:16.268049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:16.268049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:16.269177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:16.269177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:16.269177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:16.269177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:16.269177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:16.269177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:16.269177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:16.269177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:16.269177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:16.269177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:16.269177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:29:16.269177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:16.269177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 107/227] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:29:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:16.274299 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179e3e6-b608-44f8-b87d-e09c2823dc8c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:16.336105 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aba33a-9922-4a70-8394-81639960ea1b None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:16.377695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:16.395886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30edbac0-2a32-445b-b5c6-38eed0feca40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:16.396546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30edbac0-2a32-445b-b5c6-38eed0feca40 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:16.396819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30edbac0-2a32-445b-b5c6-38eed0feca40 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:16.397087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30edbac0-2a32-445b-b5c6-38eed0feca40 None admin] PATH_INFO: `/v3/users/499e4611aa4b4699aaa619a5f072bd87` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:16.398294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30edbac0-2a32-445b-b5c6-38eed0feca40 None admin] RBAC: Authorizing `identity:get_user(user_id=499e4611aa4b4699aaa619a5f072bd87)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:16.399817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30edbac0-2a32-445b-b5c6-38eed0feca40 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:16.401921 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 121/228] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:16 2022] GET /identity/v3/users/499e4611aa4b4699aaa619a5f072bd87 => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:16.407375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30edbac0-2a32-445b-b5c6-38eed0feca40 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:16.475883 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179e3e6-b608-44f8-b87d-e09c2823dc8c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:16.535148 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:16.551740 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4c9c8-852d-458d-9c32-4768047fe1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:16.552792 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4c9c8-852d-458d-9c32-4768047fe1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:16.553366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4c9c8-852d-458d-9c32-4768047fe1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:16.553877 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4c9c8-852d-458d-9c32-4768047fe1f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:16.554855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4c9c8-852d-458d-9c32-4768047fe1f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:16.555526 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4c9c8-852d-458d-9c32-4768047fe1f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:16.557149 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4c9c8-852d-458d-9c32-4768047fe1f2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:16.576914 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 108/229] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:16.578712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30edbac0-2a32-445b-b5c6-38eed0feca40 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:16.583493 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4c9c8-852d-458d-9c32-4768047fe1f2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:16.615955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a937f-dde5-4472-8a74-65a23be3b316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:16.616616 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a937f-dde5-4472-8a74-65a23be3b316 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:16.616897 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a937f-dde5-4472-8a74-65a23be3b316 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:16.617191 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a937f-dde5-4472-8a74-65a23be3b316 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:16.618365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a937f-dde5-4472-8a74-65a23be3b316 None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:16.619853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a937f-dde5-4472-8a74-65a23be3b316 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:16.621853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 122/230] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:16 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:16.630679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a937f-dde5-4472-8a74-65a23be3b316 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:16.741551 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:16.784787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a937f-dde5-4472-8a74-65a23be3b316 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:16.822043 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf92bf-dbe3-4b27-830a-5ab50b6a83e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:16.822734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf92bf-dbe3-4b27-830a-5ab50b6a83e0 None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:16.823012 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf92bf-dbe3-4b27-830a-5ab50b6a83e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:16.823532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf92bf-dbe3-4b27-830a-5ab50b6a83e0 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce/users/499e4611aa4b4699aaa619a5f072bd87/roles/05e78a4a009c41488cf3d2c47e8cbe30` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:16.826284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf92bf-dbe3-4b27-830a-5ab50b6a83e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b13122687f1640669b462d728fc006ce, user_id=499e4611aa4b4699aaa619a5f072bd87, role_id=05e78a4a009c41488cf3d2c47e8cbe30)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:16.828222 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf92bf-dbe3-4b27-830a-5ab50b6a83e0 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:16.850275 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 123/231] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:29:16 2022] PUT /identity/v3/projects/b13122687f1640669b462d728fc006ce/users/499e4611aa4b4699aaa619a5f072bd87/roles/05e78a4a009c41488cf3d2c47e8cbe30 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:16.853070 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4c9c8-852d-458d-9c32-4768047fe1f2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:16.899768 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f27020-7e1c-4cc3-bbf8-5b6f23984d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:16.900426 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f27020-7e1c-4cc3-bbf8-5b6f23984d77 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:16.900687 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f27020-7e1c-4cc3-bbf8-5b6f23984d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:16.901014 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f27020-7e1c-4cc3-bbf8-5b6f23984d77 None admin] PATH_INFO: `/v3/users/499e4611aa4b4699aaa619a5f072bd87` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:16.902217 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f27020-7e1c-4cc3-bbf8-5b6f23984d77 None admin] RBAC: Authorizing `identity:get_user(user_id=499e4611aa4b4699aaa619a5f072bd87)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:16.903785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f27020-7e1c-4cc3-bbf8-5b6f23984d77 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:16.905819 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 109/232] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:16 2022] GET /identity/v3/users/499e4611aa4b4699aaa619a5f072bd87 => generated 283 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:16.911740 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf92bf-dbe3-4b27-830a-5ab50b6a83e0 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:17.053026 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf92bf-dbe3-4b27-830a-5ab50b6a83e0 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:17.078025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:17.106136 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb47da-1c6c-44c4-af9b-7cb8c9345e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:17.107240 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb47da-1c6c-44c4-af9b-7cb8c9345e99 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:17.107773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb47da-1c6c-44c4-af9b-7cb8c9345e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:17.108237 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb47da-1c6c-44c4-af9b-7cb8c9345e99 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:17.109812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb47da-1c6c-44c4-af9b-7cb8c9345e99 None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:17.111604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb47da-1c6c-44c4-af9b-7cb8c9345e99 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:17.114033 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 124/233] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:16 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:17.120842 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f27020-7e1c-4cc3-bbf8-5b6f23984d77 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:17.206441 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:17.273112 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f27020-7e1c-4cc3-bbf8-5b6f23984d77 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:17.326607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d333e3-0bd9-4bfc-8917-c0888dbf13a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:17.327315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d333e3-0bd9-4bfc-8917-c0888dbf13a5 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:17.327596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d333e3-0bd9-4bfc-8917-c0888dbf13a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:17.327872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d333e3-0bd9-4bfc-8917-c0888dbf13a5 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce/users/499e4611aa4b4699aaa619a5f072bd87/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:17.332035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d333e3-0bd9-4bfc-8917-c0888dbf13a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b13122687f1640669b462d728fc006ce, user_id=499e4611aa4b4699aaa619a5f072bd87, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:17.333510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d333e3-0bd9-4bfc-8917-c0888dbf13a5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:17.352808 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 110/234] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:29:17 2022] PUT /identity/v3/projects/b13122687f1640669b462d728fc006ce/users/499e4611aa4b4699aaa619a5f072bd87/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:17.754113 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba9381-97ba-4858-b098-9e18a08d6898 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:17.754821 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba9381-97ba-4858-b098-9e18a08d6898 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:17.755320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba9381-97ba-4858-b098-9e18a08d6898 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:17.756969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 125/235] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:17.763183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1546ab-a5d4-4469-a81a-363ab82b0245 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:17.763800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1546ab-a5d4-4469-a81a-363ab82b0245 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:17.764269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1546ab-a5d4-4469-a81a-363ab82b0245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:17.789910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ad1546ab-a5d4-4469-a81a-363ab82b0245 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:17.791501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ad1546ab-a5d4-4469-a81a-363ab82b0245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:17.891569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 111/236] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:18.317636 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba9381-97ba-4858-b098-9e18a08d6898 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:18.334943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c708298-786c-45b0-a558-19fedb9bbbfc None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:18.335977 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c708298-786c-45b0-a558-19fedb9bbbfc None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:18.336447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c708298-786c-45b0-a558-19fedb9bbbfc None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:18.338462 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 112/237] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:18.344702 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e79401-fffe-4e77-a083-84ac72a477a2 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:18.345394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e79401-fffe-4e77-a083-84ac72a477a2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:18.345864 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e79401-fffe-4e77-a083-84ac72a477a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:18.390312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-64e79401-fffe-4e77-a083-84ac72a477a2 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:18.391905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-64e79401-fffe-4e77-a083-84ac72a477a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:18.451741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba9381-97ba-4858-b098-9e18a08d6898 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:18.496323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:18.498305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ec9dc-8bd8-44ab-bbf7-ca1fb28c4b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:18.498976 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ec9dc-8bd8-44ab-bbf7-ca1fb28c4b9b None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:18.499257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ec9dc-8bd8-44ab-bbf7-ca1fb28c4b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:18.500037 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ec9dc-8bd8-44ab-bbf7-ca1fb28c4b9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:18.514616 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ec9dc-8bd8-44ab-bbf7-ca1fb28c4b9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:18.521452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-9c9ec9dc-8bd8-44ab-bbf7-ca1fb28c4b9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:18.521452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:18.521452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:18.521452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:18.521452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:18.521452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:18.521452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:18.521452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:18.521452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:18.521452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:18.521452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:18.521452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:18.521452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:18.521452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:18.521452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:18.521452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:18.522400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:18.522400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:18.522400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:18.522400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:18.522400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:18.522400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:18.522400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:18.522400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:18.522400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:18.522400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:18.522400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:18.522400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:18.522400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:18.522400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:18.522400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:18.522400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:18.523200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:18.523200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:18.523200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:18.523200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:18.523200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:18.523200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:18.523200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:18.523200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:18.523200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:18.523200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:18.523200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:18.523200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:18.523200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 126/238] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:18.534379 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ec9dc-8bd8-44ab-bbf7-ca1fb28c4b9b None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:18.542736 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 113/239] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:29:18.544203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fc9f7-9180-4b56-bc9d-381b9cb79fa2 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:18.544478 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fc9f7-9180-4b56-bc9d-381b9cb79fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:18.544754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fc9f7-9180-4b56-bc9d-381b9cb79fa2 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:18.546586 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 114/240] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:18.552472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055bd98-3121-42dc-a733-33cae90e8a49 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:18.552806 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055bd98-3121-42dc-a733-33cae90e8a49 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:18.552993 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055bd98-3121-42dc-a733-33cae90e8a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:18.573667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a055bd98-3121-42dc-a733-33cae90e8a49 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:18.575257 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a055bd98-3121-42dc-a733-33cae90e8a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:18.596965 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:18.661531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 115/241] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:18.668590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ec9dc-8bd8-44ab-bbf7-ca1fb28c4b9b None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:18.713524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec2938-d328-44d5-80b3-1447709f4096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:18.713728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec2938-d328-44d5-80b3-1447709f4096 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:18.716337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec2938-d328-44d5-80b3-1447709f4096 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:18.716337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec2938-d328-44d5-80b3-1447709f4096 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:18.716337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec2938-d328-44d5-80b3-1447709f4096 None admin] RBAC: Adding query filter params (name=member) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:18.716337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec2938-d328-44d5-80b3-1447709f4096 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:18.716900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec2938-d328-44d5-80b3-1447709f4096 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:18.736165 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 127/242] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:18.742325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055bd98-3121-42dc-a733-33cae90e8a49 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:18.924850 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055bd98-3121-42dc-a733-33cae90e8a49 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:18.939188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:18.977558 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557e5e6-93d2-472b-9531-0bb013dbe74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:18.978214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557e5e6-93d2-472b-9531-0bb013dbe74f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:18.978492 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557e5e6-93d2-472b-9531-0bb013dbe74f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:18.978767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557e5e6-93d2-472b-9531-0bb013dbe74f None admin] PATH_INFO: `/v3/users/5f1ee136fc19419f9b4c3636e2125803` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:18.986007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557e5e6-93d2-472b-9531-0bb013dbe74f None admin] RBAC: Authorizing `identity:get_user(user_id=5f1ee136fc19419f9b4c3636e2125803)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:18.987510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557e5e6-93d2-472b-9531-0bb013dbe74f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:18.989638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 116/243] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:18 2022] GET /identity/v3/users/5f1ee136fc19419f9b4c3636e2125803 => generated 297 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:18.995059 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec2938-d328-44d5-80b3-1447709f4096 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:19.004795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557e5e6-93d2-472b-9531-0bb013dbe74f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:19.144820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec2938-d328-44d5-80b3-1447709f4096 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:19.147992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557e5e6-93d2-472b-9531-0bb013dbe74f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:19.160168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:19.193222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98194c3-98b8-4b91-82dd-7d6cbbb9abe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:19.193978 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98194c3-98b8-4b91-82dd-7d6cbbb9abe2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.194351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98194c3-98b8-4b91-82dd-7d6cbbb9abe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.194717 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98194c3-98b8-4b91-82dd-7d6cbbb9abe2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.196102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98194c3-98b8-4b91-82dd-7d6cbbb9abe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:19.197873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76814315-bcbc-42ef-a37e-098f94094e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:19.198541 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76814315-bcbc-42ef-a37e-098f94094e48 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.198821 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76814315-bcbc-42ef-a37e-098f94094e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.199094 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76814315-bcbc-42ef-a37e-098f94094e48 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.199471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98194c3-98b8-4b91-82dd-7d6cbbb9abe2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:19.200418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76814315-bcbc-42ef-a37e-098f94094e48 None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:19.202061 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76814315-bcbc-42ef-a37e-098f94094e48 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:19.202550 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 117/244] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:29:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:19.203549 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98194c3-98b8-4b91-82dd-7d6cbbb9abe2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:19.203787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 128/245] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:18 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:19.209453 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76814315-bcbc-42ef-a37e-098f94094e48 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:19.280495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:19.356076 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98194c3-98b8-4b91-82dd-7d6cbbb9abe2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:19.358389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76814315-bcbc-42ef-a37e-098f94094e48 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:19.402070 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b034d6c-fade-4c34-9b55-3e2ee6342064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:19.403118 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b034d6c-fade-4c34-9b55-3e2ee6342064 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.403771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b034d6c-fade-4c34-9b55-3e2ee6342064 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.404219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b034d6c-fade-4c34-9b55-3e2ee6342064 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.405084 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b034d6c-fade-4c34-9b55-3e2ee6342064 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:19.406943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94665b0b-3e8a-49ec-80f5-3babe8c1341a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:19.407411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b034d6c-fade-4c34-9b55-3e2ee6342064 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:19.407604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94665b0b-3e8a-49ec-80f5-3babe8c1341a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.407881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94665b0b-3e8a-49ec-80f5-3babe8c1341a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.408150 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94665b0b-3e8a-49ec-80f5-3babe8c1341a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.423786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94665b0b-3e8a-49ec-80f5-3babe8c1341a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:19.432196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-94665b0b-3e8a-49ec-80f5-3babe8c1341a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:29:19.432196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:19.432196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:19.432196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:19.432196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:19.432196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:19.432196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:19.432196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:19.432196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:19.432196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:19.432196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:19.432196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:19.432196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:19.432196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:19.432196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:19.432196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:19.433238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:19.433238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:19.433238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:19.433238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:19.433238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:19.433238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:19.433238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:19.433238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:19.433238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:19.433238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:19.433238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:19.433238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:19.433238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:19.433238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:19.433238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:19.433238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:19.440106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:19.440106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:19.440106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:19.440106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:19.440106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:19.440106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:19.440106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:19.440106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:19.440106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:19.440106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:19.440106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:29:19.440106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:19.440106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 118/246] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:19.440106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94665b0b-3e8a-49ec-80f5-3babe8c1341a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:19.451451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 129/247] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:29:19 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:19.452574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b034d6c-fade-4c34-9b55-3e2ee6342064 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:19.563699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:19.604693 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94665b0b-3e8a-49ec-80f5-3babe8c1341a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:19.609463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:19.697619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef714cbf-da70-4c51-a209-1a31546106d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:19.698672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef714cbf-da70-4c51-a209-1a31546106d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.699081 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef714cbf-da70-4c51-a209-1a31546106d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.699478 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef714cbf-da70-4c51-a209-1a31546106d1 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce/users/5f1ee136fc19419f9b4c3636e2125803/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.704684 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef714cbf-da70-4c51-a209-1a31546106d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=b13122687f1640669b462d728fc006ce, user_id=5f1ee136fc19419f9b4c3636e2125803, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:19.706116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef714cbf-da70-4c51-a209-1a31546106d1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:19.734514 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 119/248] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:29:19 2022] PUT /identity/v3/projects/b13122687f1640669b462d728fc006ce/users/5f1ee136fc19419f9b4c3636e2125803/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:19.738433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b034d6c-fade-4c34-9b55-3e2ee6342064 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:19.782009 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c76ec7-c1c3-4fd6-835f-ce0ca9e15854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:19.782752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c76ec7-c1c3-4fd6-835f-ce0ca9e15854 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:19.783094 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c76ec7-c1c3-4fd6-835f-ce0ca9e15854 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:19.783464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c76ec7-c1c3-4fd6-835f-ce0ca9e15854 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:19.784238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c76ec7-c1c3-4fd6-835f-ce0ca9e15854 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:19.784666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c76ec7-c1c3-4fd6-835f-ce0ca9e15854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:19.786367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c76ec7-c1c3-4fd6-835f-ce0ca9e15854 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:19.804733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 130/249] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:19.813846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef714cbf-da70-4c51-a209-1a31546106d1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:19.952279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef714cbf-da70-4c51-a209-1a31546106d1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:20.005706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36700b9-9780-47b3-81f2-3c61f23d6800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:20.006710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36700b9-9780-47b3-81f2-3c61f23d6800 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:20.006986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36700b9-9780-47b3-81f2-3c61f23d6800 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:20.007252 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36700b9-9780-47b3-81f2-3c61f23d6800 None admin] PATH_INFO: `/v3/users/2666f4d720464cf2a8e3a64941933bb4` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:20.026132 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b5959-5f0b-40d5-909c-c48afe51fd7d None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:20.026387 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b5959-5f0b-40d5-909c-c48afe51fd7d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:20.026664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b5959-5f0b-40d5-909c-c48afe51fd7d None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:20.031931 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 131/250] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:29:20.032158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36700b9-9780-47b3-81f2-3c61f23d6800 None admin] RBAC: Authorizing `identity:get_user(user_id=2666f4d720464cf2a8e3a64941933bb4)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:20.032158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36700b9-9780-47b3-81f2-3c61f23d6800 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:20.033571 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 120/251] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:19 2022] GET /identity/v3/users/2666f4d720464cf2a8e3a64941933bb4 => generated 297 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:20.040224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06eb69-add5-4d18-b1b3-ac4cf81e625d None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:20.040641 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06eb69-add5-4d18-b1b3-ac4cf81e625d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:20.041016 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06eb69-add5-4d18-b1b3-ac4cf81e625d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:20.045646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36700b9-9780-47b3-81f2-3c61f23d6800 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:20.069980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-4f06eb69-add5-4d18-b1b3-ac4cf81e625d None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:20.071235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4f06eb69-add5-4d18-b1b3-ac4cf81e625d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:20.104417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:20.190538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 132/252] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:20.256771 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36700b9-9780-47b3-81f2-3c61f23d6800 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:20.317272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559044cf-14f4-414c-9372-de2dab874fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:20.318234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559044cf-14f4-414c-9372-de2dab874fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:20.318630 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559044cf-14f4-414c-9372-de2dab874fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:20.319008 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559044cf-14f4-414c-9372-de2dab874fe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:20.461219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559044cf-14f4-414c-9372-de2dab874fe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:20.464256 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559044cf-14f4-414c-9372-de2dab874fe3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:20.513061 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:20.520364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06eb69-add5-4d18-b1b3-ac4cf81e625d None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:20.591587 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:20.665920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 121/253] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 05:29:20.668228 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05ad0e-4432-4889-82a6-612be7a134b0 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:20.668820 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05ad0e-4432-4889-82a6-612be7a134b0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:20.673742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05ad0e-4432-4889-82a6-612be7a134b0 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:20.676605 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 122/254] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:20 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:20.681519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05ad0e-4432-4889-82a6-612be7a134b0 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:20.737532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06eb69-add5-4d18-b1b3-ac4cf81e625d None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:20.834041 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb6885-2bf2-4bbe-8943-871d73803405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:20.834707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb6885-2bf2-4bbe-8943-871d73803405 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:20.834983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb6885-2bf2-4bbe-8943-871d73803405 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:20.837380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb6885-2bf2-4bbe-8943-871d73803405 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:20.863071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb6885-2bf2-4bbe-8943-871d73803405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:20.872635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-e9cb6885-2bf2-4bbe-8943-871d73803405 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:20.872635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:20.872635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:20.872635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:20.872635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:20.872635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:20.872635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:20.872635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:20.872635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:20.872635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:20.872635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:20.872635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:20.872635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:20.872635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:20.872635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:20.872635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:20.873627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:20.873627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:20.873627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:20.873627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:20.873627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:20.873627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:20.873627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:20.873627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:20.873627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:20.873627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:20.873627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:20.873627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:20.873627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:20.873627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:20.873627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:20.873627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:20.874712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:20.874712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:20.874712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:20.874712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:20.874712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:20.874712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:20.874712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:20.874712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:20.874712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:20.874712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:20.874712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:20.874712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:20.874712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 133/255] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:29:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:20.875456 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e372a3-4e63-4fde-a81f-e223edffa0e3 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:20.876169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e372a3-4e63-4fde-a81f-e223edffa0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:20.876169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e372a3-4e63-4fde-a81f-e223edffa0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:20.920277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-08e372a3-4e63-4fde-a81f-e223edffa0e3 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:20.921524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-08e372a3-4e63-4fde-a81f-e223edffa0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:20.927752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05ad0e-4432-4889-82a6-612be7a134b0 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:20.976456 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:21.005828 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531cda3-0196-4296-9cd1-ff7532e10596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:21.006891 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531cda3-0196-4296-9cd1-ff7532e10596 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:21.007375 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531cda3-0196-4296-9cd1-ff7532e10596 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:21.007839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531cda3-0196-4296-9cd1-ff7532e10596 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:21.013820 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531cda3-0196-4296-9cd1-ff7532e10596 None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:21.016213 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531cda3-0196-4296-9cd1-ff7532e10596 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:21.028281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 123/256] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:20 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:21.029784 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531cda3-0196-4296-9cd1-ff7532e10596 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:21.096490 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 134/257] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:21.097328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e372a3-4e63-4fde-a81f-e223edffa0e3 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:21.120825 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:21.214149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e372a3-4e63-4fde-a81f-e223edffa0e3 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:21.256798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042858f-bf62-4b19-a95d-73cc15affd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:21.257761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042858f-bf62-4b19-a95d-73cc15affd53 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:21.258222 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042858f-bf62-4b19-a95d-73cc15affd53 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:21.258642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042858f-bf62-4b19-a95d-73cc15affd53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:21.259572 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042858f-bf62-4b19-a95d-73cc15affd53 None admin] RBAC: Adding query filter params (role.id=27a25b7936b8463c9e6d34d2cb49e61f, scope.project.id=b13122687f1640669b462d728fc006ce, user.id=2666f4d720464cf2a8e3a64941933bb4) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:21.260055 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042858f-bf62-4b19-a95d-73cc15affd53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:21.261795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042858f-bf62-4b19-a95d-73cc15affd53 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:21.270452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 135/258] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:29:21 2022] GET /identity/v3/role_assignments?user.id=2666f4d720464cf2a8e3a64941933bb4&scope.project.id=b13122687f1640669b462d728fc006ce&role.id=27a25b7936b8463c9e6d34d2cb49e61f => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:21.322619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531cda3-0196-4296-9cd1-ff7532e10596 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:21.391810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bfe28-515b-4927-acd1-44c00aef21b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:21.392726 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bfe28-515b-4927-acd1-44c00aef21b6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:21.393170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bfe28-515b-4927-acd1-44c00aef21b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:21.393599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bfe28-515b-4927-acd1-44c00aef21b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:21.394768 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bfe28-515b-4927-acd1-44c00aef21b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:21.395292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bfe28-515b-4927-acd1-44c00aef21b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:21.397983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bfe28-515b-4927-acd1-44c00aef21b6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:21.414820 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 124/259] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:29:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:21.421187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042858f-bf62-4b19-a95d-73cc15affd53 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:21.557311 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:21.586278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042858f-bf62-4b19-a95d-73cc15affd53 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:21.637245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362dd29-7d00-458c-9c22-f67ff320e83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:21.637917 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362dd29-7d00-458c-9c22-f67ff320e83f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:21.638198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362dd29-7d00-458c-9c22-f67ff320e83f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:21.638472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362dd29-7d00-458c-9c22-f67ff320e83f None admin] PATH_INFO: `/v3/users/499e4611aa4b4699aaa619a5f072bd87` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:21.641595 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362dd29-7d00-458c-9c22-f67ff320e83f None admin] RBAC: Authorizing `identity:get_user(user_id=499e4611aa4b4699aaa619a5f072bd87)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:21.643146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362dd29-7d00-458c-9c22-f67ff320e83f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:21.645168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 136/260] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:21 2022] GET /identity/v3/users/499e4611aa4b4699aaa619a5f072bd87 => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:21.651523 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bfe28-515b-4927-acd1-44c00aef21b6 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:21.672758 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362dd29-7d00-458c-9c22-f67ff320e83f None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:21.744405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:21.869112 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362dd29-7d00-458c-9c22-f67ff320e83f None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:21.883302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bfe28-515b-4927-acd1-44c00aef21b6 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:21.908287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030fa9c-dc1a-4ee0-ad8a-99d665630245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:21.908861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030fa9c-dc1a-4ee0-ad8a-99d665630245 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:21.909348 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030fa9c-dc1a-4ee0-ad8a-99d665630245 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:21.909622 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030fa9c-dc1a-4ee0-ad8a-99d665630245 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:21.930077 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030fa9c-dc1a-4ee0-ad8a-99d665630245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:21.935938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-f030fa9c-dc1a-4ee0-ad8a-99d665630245 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:29:21.935938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:21.935938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:21.935938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:21.935938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:21.935938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:21.935938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:21.935938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:21.935938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:21.935938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:21.935938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:21.935938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:21.935938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:21.935938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:21.935938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:21.935938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:21.936872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:21.936872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:21.936872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:21.936872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:21.936872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:21.936872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:21.936872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:21.936872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:21.936872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:21.936872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:21.936872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:21.936872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:21.936872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:21.936872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:21.936872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:21.936872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:21.937798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:21.937798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:21.937798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:21.937798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:21.937798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:21.937798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:21.937798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:21.937798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:21.937798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:21.937798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:21.937798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:29:21.937798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:21.945002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 137/261] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:29:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:21.945206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf87a09-5367-46da-814a-e8588c75d588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:21.945206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf87a09-5367-46da-814a-e8588c75d588 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:21.945206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf87a09-5367-46da-814a-e8588c75d588 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:21.945206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf87a09-5367-46da-814a-e8588c75d588 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:21.946331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030fa9c-dc1a-4ee0-ad8a-99d665630245 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:21.968448 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:22.075944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf87a09-5367-46da-814a-e8588c75d588 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:22.077488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf87a09-5367-46da-814a-e8588c75d588 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:22.139667 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030fa9c-dc1a-4ee0-ad8a-99d665630245 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:22.185586 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:22.217720 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a13819-64fe-4771-a39a-4f1eb5835a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:22.218669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a13819-64fe-4771-a39a-4f1eb5835a7e None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:22.219132 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a13819-64fe-4771-a39a-4f1eb5835a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:22.219642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a13819-64fe-4771-a39a-4f1eb5835a7e None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:22.220666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a13819-64fe-4771-a39a-4f1eb5835a7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:22.222036 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a13819-64fe-4771-a39a-4f1eb5835a7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:22.226284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a13819-64fe-4771-a39a-4f1eb5835a7e None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:22.242763 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 138/262] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:22.247784 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a13819-64fe-4771-a39a-4f1eb5835a7e None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:22.282509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 125/263] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:22.293563 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf87a09-5367-46da-814a-e8588c75d588 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:22.385163 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:22.446063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a13819-64fe-4771-a39a-4f1eb5835a7e None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:22.500288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf87a09-5367-46da-814a-e8588c75d588 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:22.522519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c03af-2f53-4443-a13c-df6bcfde0384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:22.523180 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c03af-2f53-4443-a13c-df6bcfde0384 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:22.523450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c03af-2f53-4443-a13c-df6bcfde0384 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:22.523721 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c03af-2f53-4443-a13c-df6bcfde0384 None admin] PATH_INFO: `/v3/users/499e4611aa4b4699aaa619a5f072bd87` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:22.527378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c03af-2f53-4443-a13c-df6bcfde0384 None admin] RBAC: Authorizing `identity:get_user(user_id=499e4611aa4b4699aaa619a5f072bd87)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:22.530867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c03af-2f53-4443-a13c-df6bcfde0384 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:22.532602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 139/264] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:22 2022] GET /identity/v3/users/499e4611aa4b4699aaa619a5f072bd87 => generated 283 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:22.541440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c03af-2f53-4443-a13c-df6bcfde0384 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:22.587758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cba413-5812-4933-825d-98367625ce5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:22.588736 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cba413-5812-4933-825d-98367625ce5d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:22.589180 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cba413-5812-4933-825d-98367625ce5d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:22.589592 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cba413-5812-4933-825d-98367625ce5d None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:22.592660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cba413-5812-4933-825d-98367625ce5d None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:22.595438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cba413-5812-4933-825d-98367625ce5d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:22.601248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 126/265] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:22 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:22.612784 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cba413-5812-4933-825d-98367625ce5d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:22.697633 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:22.768298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c03af-2f53-4443-a13c-df6bcfde0384 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:22.865972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3ea7c-f7a5-4456-b9e1-c70ac9986c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:22.866618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3ea7c-f7a5-4456-b9e1-c70ac9986c30 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:22.866899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3ea7c-f7a5-4456-b9e1-c70ac9986c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:22.867168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3ea7c-f7a5-4456-b9e1-c70ac9986c30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:22.869959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cba413-5812-4933-825d-98367625ce5d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:22.881501 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:22.981440 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322ef11-0e48-47ef-9caa-7ed6c66e3de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:22.982383 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322ef11-0e48-47ef-9caa-7ed6c66e3de8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:22.982883 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322ef11-0e48-47ef-9caa-7ed6c66e3de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:22.983369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322ef11-0e48-47ef-9caa-7ed6c66e3de8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:22.984767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322ef11-0e48-47ef-9caa-7ed6c66e3de8 None admin] RBAC: Adding query filter params (role.id=05e78a4a009c41488cf3d2c47e8cbe30, scope.project.id=b13122687f1640669b462d728fc006ce, user.id=499e4611aa4b4699aaa619a5f072bd87) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:22.985357 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322ef11-0e48-47ef-9caa-7ed6c66e3de8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:22.988804 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322ef11-0e48-47ef-9caa-7ed6c66e3de8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:23.007211 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 127/266] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:29:22 2022] GET /identity/v3/role_assignments?user.id=499e4611aa4b4699aaa619a5f072bd87&scope.project.id=b13122687f1640669b462d728fc006ce&role.id=05e78a4a009c41488cf3d2c47e8cbe30 => generated 608 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:23.040116 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3ea7c-f7a5-4456-b9e1-c70ac9986c30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:23.043128 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3ea7c-f7a5-4456-b9e1-c70ac9986c30 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:23.269019 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 140/267] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:23.270539 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322ef11-0e48-47ef-9caa-7ed6c66e3de8 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:23.283141 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31bd64-f302-454d-85ef-4d131776226d None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:23.283458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31bd64-f302-454d-85ef-4d131776226d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:23.283690 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31bd64-f302-454d-85ef-4d131776226d None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:23.285167 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 141/268] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:29:23.298155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfb508-56fc-45a4-b7c5-f2ac0581befc None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:23.298467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfb508-56fc-45a4-b7c5-f2ac0581befc None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:23.298709 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfb508-56fc-45a4-b7c5-f2ac0581befc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:23.332761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-36dfb508-56fc-45a4-b7c5-f2ac0581befc None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:23.334114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-36dfb508-56fc-45a4-b7c5-f2ac0581befc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:23.369803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:23.421303 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 142/269] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:29:23.497696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:23.499904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322ef11-0e48-47ef-9caa-7ed6c66e3de8 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:23.543651 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31ae0f-9a86-4e0b-b4bf-105995ed6637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:23.544420 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31ae0f-9a86-4e0b-b4bf-105995ed6637 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:23.544701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31ae0f-9a86-4e0b-b4bf-105995ed6637 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:23.544976 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31ae0f-9a86-4e0b-b4bf-105995ed6637 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:23.546305 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31ae0f-9a86-4e0b-b4bf-105995ed6637 None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:23.547783 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f31ae0f-9a86-4e0b-b4bf-105995ed6637 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:23.549874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 128/270] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:23 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:23.555874 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfb508-56fc-45a4-b7c5-f2ac0581befc None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:23.678814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfb508-56fc-45a4-b7c5-f2ac0581befc None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:23.719581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00bf45-f87c-46dc-99b3-61544e5835ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:23.720228 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00bf45-f87c-46dc-99b3-61544e5835ec None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:23.720505 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00bf45-f87c-46dc-99b3-61544e5835ec None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:23.720779 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00bf45-f87c-46dc-99b3-61544e5835ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:23.721822 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00bf45-f87c-46dc-99b3-61544e5835ec None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=b13122687f1640669b462d728fc006ce, user.id=499e4611aa4b4699aaa619a5f072bd87) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:23.722141 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00bf45-f87c-46dc-99b3-61544e5835ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:23.724036 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31ae0f-9a86-4e0b-b4bf-105995ed6637 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:23.724749 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00bf45-f87c-46dc-99b3-61544e5835ec None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:23.731397 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 143/271] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:29:23 2022] GET /identity/v3/role_assignments?user.id=499e4611aa4b4699aaa619a5f072bd87&scope.project.id=b13122687f1640669b462d728fc006ce&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:23.843682 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d2aed-568b-4a33-b4fb-2710f8ed38a8 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:23.844681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d2aed-568b-4a33-b4fb-2710f8ed38a8 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:23.844681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d2aed-568b-4a33-b4fb-2710f8ed38a8 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:23.845960 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 144/272] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:29:23.852278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55f735-5310-49d4-ab93-7091943ca19a None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:23.852539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55f735-5310-49d4-ab93-7091943ca19a None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:23.852814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55f735-5310-49d4-ab93-7091943ca19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:23.876678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-ba55f735-5310-49d4-ab93-7091943ca19a None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:23.878825 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ba55f735-5310-49d4-ab93-7091943ca19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:23.937261 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:23.969856 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31ae0f-9a86-4e0b-b4bf-105995ed6637 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:23.992183 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 145/273] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:23.993750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb35d2b-56c5-4c7a-9ea2-d77b606c7345 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:23.994052 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb35d2b-56c5-4c7a-9ea2-d77b606c7345 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:23.994301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb35d2b-56c5-4c7a-9ea2-d77b606c7345 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:23.995988 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 146/274] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:24.002202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462721a-f632-4ecb-ba9b-01964e9cdb13 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:24.002478 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462721a-f632-4ecb-ba9b-01964e9cdb13 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:24.002767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462721a-f632-4ecb-ba9b-01964e9cdb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:24.034202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a603898-8cb1-410e-953c-6ef784f6cb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:24.034202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a603898-8cb1-410e-953c-6ef784f6cb14 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:24.034202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a603898-8cb1-410e-953c-6ef784f6cb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:24.034664 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a603898-8cb1-410e-953c-6ef784f6cb14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:24.037090 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a603898-8cb1-410e-953c-6ef784f6cb14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:24.038599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a603898-8cb1-410e-953c-6ef784f6cb14 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:24.040305 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 129/275] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:29:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:24.043968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-f462721a-f632-4ecb-ba9b-01964e9cdb13 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:24.046444 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a603898-8cb1-410e-953c-6ef784f6cb14 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:24.047829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f462721a-f632-4ecb-ba9b-01964e9cdb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:24.072979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:24.193197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a603898-8cb1-410e-953c-6ef784f6cb14 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:24.195665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 147/276] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:29:24.241234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915a070-aa4e-4dac-8ca2-9ade3a6b4f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:24.242271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915a070-aa4e-4dac-8ca2-9ade3a6b4f1e None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:24.242739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915a070-aa4e-4dac-8ca2-9ade3a6b4f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:24.243235 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915a070-aa4e-4dac-8ca2-9ade3a6b4f1e None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:24.244337 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915a070-aa4e-4dac-8ca2-9ade3a6b4f1e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:24.247289 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915a070-aa4e-4dac-8ca2-9ade3a6b4f1e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:24.289034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 130/277] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:29:24 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:29:24.319519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462721a-f632-4ecb-ba9b-01964e9cdb13 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:24.471721 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:24.484116 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462721a-f632-4ecb-ba9b-01964e9cdb13 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:24.528540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a06ed7-7405-47f0-9e66-5d24e3998e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:24.529229 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a06ed7-7405-47f0-9e66-5d24e3998e97 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:24.529508 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a06ed7-7405-47f0-9e66-5d24e3998e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:24.529775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a06ed7-7405-47f0-9e66-5d24e3998e97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:24.543291 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a06ed7-7405-47f0-9e66-5d24e3998e97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:24.548277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-05a06ed7-7405-47f0-9e66-5d24e3998e97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:24.548277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:24.548277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:24.548277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:24.548277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:24.548277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:24.548277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:24.548277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:24.548277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:24.548277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:24.548277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:24.548277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:24.548277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:24.548277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:24.548277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:24.548277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:24.549235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:24.549235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:24.549235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:24.549235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:24.549235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:24.549235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:24.549235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:24.549235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:24.549235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:24.549235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:24.549235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:24.549235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:24.549235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:24.549235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:24.549235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:24.549235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:24.550511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:24.550511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:24.550511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:24.550511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:24.550511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:24.550511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:24.550511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:24.550511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:24.550511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:24.550511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:24.550511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:24.550511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:24.550511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 148/278] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:24.556533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915a070-aa4e-4dac-8ca2-9ade3a6b4f1e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:24.595466 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a06ed7-7405-47f0-9e66-5d24e3998e97 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:24.672504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:24.715928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915a070-aa4e-4dac-8ca2-9ade3a6b4f1e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:24.764141 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a06ed7-7405-47f0-9e66-5d24e3998e97 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:24.772210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe95e23-cc7b-4774-8b49-54c9bd1721fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:24.774504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe95e23-cc7b-4774-8b49-54c9bd1721fd None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:24.775252 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe95e23-cc7b-4774-8b49-54c9bd1721fd None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:24.776714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe95e23-cc7b-4774-8b49-54c9bd1721fd None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:24.778353 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe95e23-cc7b-4774-8b49-54c9bd1721fd None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:24.779212 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe95e23-cc7b-4774-8b49-54c9bd1721fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:24.782134 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe95e23-cc7b-4774-8b49-54c9bd1721fd None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:24.798147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 131/279] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:24.805346 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ee44e-46ef-4f22-a68f-42b13d9e0ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:24.806001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ee44e-46ef-4f22-a68f-42b13d9e0ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:24.806337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ee44e-46ef-4f22-a68f-42b13d9e0ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:24.806603 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ee44e-46ef-4f22-a68f-42b13d9e0ed6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:24.809119 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe95e23-cc7b-4774-8b49-54c9bd1721fd None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:24.820942 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ee44e-46ef-4f22-a68f-42b13d9e0ed6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:24.826101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-e99ee44e-46ef-4f22-a68f-42b13d9e0ed6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:24.826101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:24.826101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:24.826101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:24.826101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:24.826101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:24.826101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:24.826101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:24.826101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:24.826101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:24.826101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:24.826101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:24.826101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:24.826101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:24.826101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:24.826101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:24.834807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:24.834807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:24.834807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:24.834807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:24.834807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:24.834807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:24.834807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:24.834807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:24.834807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:24.834807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:24.834807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:24.834807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:24.834807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:24.834807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:24.834807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:24.834807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:24.836280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:24.836280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:24.836280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:24.836280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:24.836280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:24.836280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:24.836280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:24.836280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:24.836280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:24.836280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:24.836280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:24.836280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:24.836280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 149/280] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:24.842971 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ee44e-46ef-4f22-a68f-42b13d9e0ed6 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:24.878730 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:25.023958 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe95e23-cc7b-4774-8b49-54c9bd1721fd None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:25.057195 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ee44e-46ef-4f22-a68f-42b13d9e0ed6 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:25.124810 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2f99b-98e2-4207-b39a-b87d9f068edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:25.125874 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2f99b-98e2-4207-b39a-b87d9f068edb None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:25.126873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2f99b-98e2-4207-b39a-b87d9f068edb None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:25.127066 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2f99b-98e2-4207-b39a-b87d9f068edb None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:25.128164 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2f99b-98e2-4207-b39a-b87d9f068edb None admin] RBAC: Adding query filter params (name=member) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:25.128727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2f99b-98e2-4207-b39a-b87d9f068edb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:25.137173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2f99b-98e2-4207-b39a-b87d9f068edb None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:25.148002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6949b8-e01e-42b8-93de-fdc3871003f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:25.148964 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 150/281] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:25.149917 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6949b8-e01e-42b8-93de-fdc3871003f1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:25.150395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6949b8-e01e-42b8-93de-fdc3871003f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:25.150866 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6949b8-e01e-42b8-93de-fdc3871003f1 None admin] PATH_INFO: `/v3/users/f411494b67af4bf0a06f2e1bfed51474` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:25.164365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2f99b-98e2-4207-b39a-b87d9f068edb None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:25.185756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:25.208048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6949b8-e01e-42b8-93de-fdc3871003f1 None admin] RBAC: Authorizing `identity:get_user(user_id=f411494b67af4bf0a06f2e1bfed51474)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:25.210892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6949b8-e01e-42b8-93de-fdc3871003f1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:25.214336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 132/282] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:24 2022] GET /identity/v3/users/f411494b67af4bf0a06f2e1bfed51474 => generated 293 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:25.230965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6949b8-e01e-42b8-93de-fdc3871003f1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:25.365573 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:25.421038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2f99b-98e2-4207-b39a-b87d9f068edb None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:25.488484 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6949b8-e01e-42b8-93de-fdc3871003f1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:25.491005 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932c3aa-686e-4a5a-b749-ca97396e844d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:25.492366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932c3aa-686e-4a5a-b749-ca97396e844d None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:25.492839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932c3aa-686e-4a5a-b749-ca97396e844d None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:25.494742 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932c3aa-686e-4a5a-b749-ca97396e844d None admin] PATH_INFO: `/v3/users/5f1ee136fc19419f9b4c3636e2125803` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:25.497014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932c3aa-686e-4a5a-b749-ca97396e844d None admin] RBAC: Authorizing `identity:get_user(user_id=5f1ee136fc19419f9b4c3636e2125803)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:25.499901 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932c3aa-686e-4a5a-b749-ca97396e844d None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:25.505396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 151/283] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:25 2022] GET /identity/v3/users/5f1ee136fc19419f9b4c3636e2125803 => generated 297 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:25.507672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978a2de-bf37-458e-8707-3c0cea999e8d None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:25.508140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978a2de-bf37-458e-8707-3c0cea999e8d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:25.508594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978a2de-bf37-458e-8707-3c0cea999e8d None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:25.515486 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 152/284] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:25.517532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978a2de-bf37-458e-8707-3c0cea999e8d None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:25.566753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f0c0b-857d-451f-afc6-bbf08be71136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:25.567988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f0c0b-857d-451f-afc6-bbf08be71136 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:25.568432 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f0c0b-857d-451f-afc6-bbf08be71136 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:25.569438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f0c0b-857d-451f-afc6-bbf08be71136 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:25.570222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f0c0b-857d-451f-afc6-bbf08be71136 None admin] RBAC: Adding query filter params (role.id=bb454faf5dd643b892c6f9b670ad1f2d, user.id=f411494b67af4bf0a06f2e1bfed51474, scope.system=all) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:25.570530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f0c0b-857d-451f-afc6-bbf08be71136 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:25.572049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f0c0b-857d-451f-afc6-bbf08be71136 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:25.585181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 133/285] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Sep 26 05:29:25 2022] GET /identity/v3/role_assignments?user.id=f411494b67af4bf0a06f2e1bfed51474&scope.system=all&role.id=bb454faf5dd643b892c6f9b670ad1f2d => generated 226 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:25.585181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de208cb-bd86-41ee-af6e-9e2e3f78aa17 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:25.585181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de208cb-bd86-41ee-af6e-9e2e3f78aa17 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:25.585181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de208cb-bd86-41ee-af6e-9e2e3f78aa17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:25.622587 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-3de208cb-bd86-41ee-af6e-9e2e3f78aa17 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:25.623830 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3de208cb-bd86-41ee-af6e-9e2e3f78aa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:25.705009 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978a2de-bf37-458e-8707-3c0cea999e8d None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:25.764596 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f244050-13ec-4fa4-b1ef-9d939a80ffc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:25.765663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f244050-13ec-4fa4-b1ef-9d939a80ffc9 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:25.767458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f244050-13ec-4fa4-b1ef-9d939a80ffc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:25.768010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f244050-13ec-4fa4-b1ef-9d939a80ffc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:25.780056 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:25.798679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 134/286] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:25.812358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:25.960048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f244050-13ec-4fa4-b1ef-9d939a80ffc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:25.964022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f244050-13ec-4fa4-b1ef-9d939a80ffc9 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:26.081899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de208cb-bd86-41ee-af6e-9e2e3f78aa17 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:26.163919 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 153/287] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:26.169960 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f244050-13ec-4fa4-b1ef-9d939a80ffc9 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:26.257625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:26.345897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de208cb-bd86-41ee-af6e-9e2e3f78aa17 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:26.416903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f244050-13ec-4fa4-b1ef-9d939a80ffc9 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:26.419841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d686ba-b41e-4886-9cb9-25a61c271dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:26.420932 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d686ba-b41e-4886-9cb9-25a61c271dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:26.421395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d686ba-b41e-4886-9cb9-25a61c271dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:26.421880 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d686ba-b41e-4886-9cb9-25a61c271dd1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:26.460097 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d686ba-b41e-4886-9cb9-25a61c271dd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:26.466510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-a3d686ba-b41e-4886-9cb9-25a61c271dd1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:29:26.466510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:26.466510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:26.466510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:26.466510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:26.466510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:26.466510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:26.466510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:26.466510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:26.466510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:26.466510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:26.466510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:26.466510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:26.466510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:26.466510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:26.466510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:26.467438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:26.467438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:26.467438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:26.467438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:26.467438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:26.467438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:26.467438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:26.467438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:26.467438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:26.467438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:26.467438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:26.467438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:26.467438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:26.467438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:26.467438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:26.467438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:26.468220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:26.468220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:26.468220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:26.468220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:26.468220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:26.468220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:26.468220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:26.468220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:26.468220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:26.468220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:26.468220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:29:26.468220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:26.468220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 135/288] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:26.478404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75954eb4-e356-4a78-9e2e-f623453e91bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:26.479470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75954eb4-e356-4a78-9e2e-f623453e91bf None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:26.479945 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75954eb4-e356-4a78-9e2e-f623453e91bf None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:26.480411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75954eb4-e356-4a78-9e2e-f623453e91bf None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:26.481912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75954eb4-e356-4a78-9e2e-f623453e91bf None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:26.482480 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d686ba-b41e-4886-9cb9-25a61c271dd1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:26.483982 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75954eb4-e356-4a78-9e2e-f623453e91bf None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:26.486331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 154/289] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:26 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:26.495277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75954eb4-e356-4a78-9e2e-f623453e91bf None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:26.627110 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:26.703925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75954eb4-e356-4a78-9e2e-f623453e91bf None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:26.745657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:26.759327 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d686ba-b41e-4886-9cb9-25a61c271dd1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:26.803902 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34417f5-2560-4c31-9624-a0af8517a1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:26.804807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34417f5-2560-4c31-9624-a0af8517a1c8 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:26.805239 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34417f5-2560-4c31-9624-a0af8517a1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:26.805643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34417f5-2560-4c31-9624-a0af8517a1c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:26.806800 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34417f5-2560-4c31-9624-a0af8517a1c8 None admin] RBAC: Adding query filter params (role.id=bb454faf5dd643b892c6f9b670ad1f2d, scope.project.id=b13122687f1640669b462d728fc006ce, user.id=5f1ee136fc19419f9b4c3636e2125803) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:26.807314 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34417f5-2560-4c31-9624-a0af8517a1c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:26.812268 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34417f5-2560-4c31-9624-a0af8517a1c8 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:26.846370 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 155/290] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:29:26 2022] GET /identity/v3/role_assignments?user.id=5f1ee136fc19419f9b4c3636e2125803&scope.project.id=b13122687f1640669b462d728fc006ce&role.id=bb454faf5dd643b892c6f9b670ad1f2d => generated 608 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:26.870692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c7403-baf1-4ad3-a15c-943c83d79b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:26.871723 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c7403-baf1-4ad3-a15c-943c83d79b83 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:26.872141 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c7403-baf1-4ad3-a15c-943c83d79b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:26.872545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c7403-baf1-4ad3-a15c-943c83d79b83 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:26.873711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c7403-baf1-4ad3-a15c-943c83d79b83 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:26.874163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c7403-baf1-4ad3-a15c-943c83d79b83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:26.876619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c7403-baf1-4ad3-a15c-943c83d79b83 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:26.900137 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 136/291] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:26.906771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34417f5-2560-4c31-9624-a0af8517a1c8 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:27.125318 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34417f5-2560-4c31-9624-a0af8517a1c8 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:27.188701 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5833445-6a41-40d6-ad45-5edc0f031a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:27.189799 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5833445-6a41-40d6-ad45-5edc0f031a6b None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:27.190271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5833445-6a41-40d6-ad45-5edc0f031a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:27.190725 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5833445-6a41-40d6-ad45-5edc0f031a6b None admin] PATH_INFO: `/v3/users/2666f4d720464cf2a8e3a64941933bb4` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:27.192842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5833445-6a41-40d6-ad45-5edc0f031a6b None admin] RBAC: Authorizing `identity:get_user(user_id=2666f4d720464cf2a8e3a64941933bb4)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:27.195576 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5833445-6a41-40d6-ad45-5edc0f031a6b None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:27.199075 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 156/292] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:26 2022] GET /identity/v3/users/2666f4d720464cf2a8e3a64941933bb4 => generated 297 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:27.205576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c7403-baf1-4ad3-a15c-943c83d79b83 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:27.359818 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c7403-baf1-4ad3-a15c-943c83d79b83 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:27.378233 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:27.415006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f66262-dfd1-4932-9387-8cb7ca738728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:27.415656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f66262-dfd1-4932-9387-8cb7ca738728 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:27.415931 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f66262-dfd1-4932-9387-8cb7ca738728 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:27.416199 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f66262-dfd1-4932-9387-8cb7ca738728 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:27.417500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f66262-dfd1-4932-9387-8cb7ca738728 None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:27.419072 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f66262-dfd1-4932-9387-8cb7ca738728 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:27.421118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 137/293] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:27 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:27.428160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5833445-6a41-40d6-ad45-5edc0f031a6b None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:27.519485 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:27.605971 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5078cb-4a2d-433f-baba-b74e15d77d11 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:27.606307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5078cb-4a2d-433f-baba-b74e15d77d11 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:27.606530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5078cb-4a2d-433f-baba-b74e15d77d11 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:27.608136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 138/294] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:27.618082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08231f5a-2f55-4439-8268-ccd38f792757 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:27.618417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08231f5a-2f55-4439-8268-ccd38f792757 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:27.618652 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08231f5a-2f55-4439-8268-ccd38f792757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:27.638545 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5833445-6a41-40d6-ad45-5edc0f031a6b None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:27.640728 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-08231f5a-2f55-4439-8268-ccd38f792757 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:27.642365 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-08231f5a-2f55-4439-8268-ccd38f792757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:27.707625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf056c9-0d0f-4e0d-88f9-e47f8a075e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:27.708419 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf056c9-0d0f-4e0d-88f9-e47f8a075e86 None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:27.708700 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf056c9-0d0f-4e0d-88f9-e47f8a075e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:27.708971 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf056c9-0d0f-4e0d-88f9-e47f8a075e86 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce/users/2666f4d720464cf2a8e3a64941933bb4/roles/27a25b7936b8463c9e6d34d2cb49e61f` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:27.711467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf056c9-0d0f-4e0d-88f9-e47f8a075e86 None admin] RBAC: Authorizing `identity:create_grant(project_id=b13122687f1640669b462d728fc006ce, user_id=2666f4d720464cf2a8e3a64941933bb4, role_id=27a25b7936b8463c9e6d34d2cb49e61f)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:27.713777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf056c9-0d0f-4e0d-88f9-e47f8a075e86 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:27.730504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 139/295] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:29:27.731806 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb053de-5f7b-4754-84d0-4bc087307fee None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:27.732093 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb053de-5f7b-4754-84d0-4bc087307fee None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:27.732360 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb053de-5f7b-4754-84d0-4bc087307fee None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:27.733775 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 140/296] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:27.744411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 157/297] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:29:27 2022] PUT /identity/v3/projects/b13122687f1640669b462d728fc006ce/users/2666f4d720464cf2a8e3a64941933bb4/roles/27a25b7936b8463c9e6d34d2cb49e61f => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:27.748110 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd48117-6c60-4825-8d4d-7cd322a117a5 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:27.749359 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd48117-6c60-4825-8d4d-7cd322a117a5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:27.750233 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd48117-6c60-4825-8d4d-7cd322a117a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:27.786042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-6cd48117-6c60-4825-8d4d-7cd322a117a5 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:27.788839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6cd48117-6c60-4825-8d4d-7cd322a117a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:27.915463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 141/298] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:28.042931 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf056c9-0d0f-4e0d-88f9-e47f8a075e86 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:28.155953 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf056c9-0d0f-4e0d-88f9-e47f8a075e86 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:28.164196 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:28.196396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120142a4-ef3b-4fff-a94d-ca68efde74fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:28.196992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120142a4-ef3b-4fff-a94d-ca68efde74fc None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:28.197302 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120142a4-ef3b-4fff-a94d-ca68efde74fc None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:28.197574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120142a4-ef3b-4fff-a94d-ca68efde74fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:28.198767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120142a4-ef3b-4fff-a94d-ca68efde74fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:28.200322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120142a4-ef3b-4fff-a94d-ca68efde74fc None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:28.202368 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 158/299] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:29:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:28.208241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd48117-6c60-4825-8d4d-7cd322a117a5 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:28.257297 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:28.346579 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd48117-6c60-4825-8d4d-7cd322a117a5 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:28.374265 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120142a4-ef3b-4fff-a94d-ca68efde74fc None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:28.410352 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c4eae-959b-41b3-be05-b9a55e7bff22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:28.410983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c4eae-959b-41b3-be05-b9a55e7bff22 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:28.411301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c4eae-959b-41b3-be05-b9a55e7bff22 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:28.411553 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c4eae-959b-41b3-be05-b9a55e7bff22 None admin] PATH_INFO: `/v3/groups` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:28.412160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c4eae-959b-41b3-be05-b9a55e7bff22 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:28.414181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c4eae-959b-41b3-be05-b9a55e7bff22 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:28.445352 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 142/300] 10.222.0.57 () {60 vars in 1134 bytes} [Mon Sep 26 05:29:28 2022] POST /identity/v3/groups => generated 229 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:28.532271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120142a4-ef3b-4fff-a94d-ca68efde74fc None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:28.578765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3437a32-cce6-480b-ad66-6a8553abb553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:28.579871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3437a32-cce6-480b-ad66-6a8553abb553 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:28.580390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3437a32-cce6-480b-ad66-6a8553abb553 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:28.580854 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3437a32-cce6-480b-ad66-6a8553abb553 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:28.594796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3437a32-cce6-480b-ad66-6a8553abb553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:28.601441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-a3437a32-cce6-480b-ad66-6a8553abb553 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:29:28.601441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:28.601441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:28.601441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:28.601441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:28.601441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:28.601441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:28.601441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:28.601441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:28.601441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:28.601441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:28.601441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:28.601441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:28.601441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:28.601441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:28.601441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:28.602404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:28.602404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:28.602404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:28.602404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:28.602404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:28.602404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:28.602404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:28.602404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:28.602404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:28.602404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:28.602404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:28.602404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:28.602404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:28.602404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:28.602404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:28.602404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:28.603194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:28.603194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:28.603194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:28.603194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:28.603194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:28.603194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:28.603194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:28.603194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:28.603194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:28.603194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:28.603194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:29:28.603194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:28.606342 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 159/301] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:29:28.615182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7413b-c5d9-47b6-81a4-35290447b2f8 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:28.615482 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3437a32-cce6-480b-ad66-6a8553abb553 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:28.615615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7413b-c5d9-47b6-81a4-35290447b2f8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:28.615953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7413b-c5d9-47b6-81a4-35290447b2f8 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:28.617929 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 143/302] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:28.625744 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f18d7-8cb0-4c6f-b1d5-189747d0cd79 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:28.626041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f18d7-8cb0-4c6f-b1d5-189747d0cd79 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:28.626345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f18d7-8cb0-4c6f-b1d5-189747d0cd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:28.647184 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-0a1f18d7-8cb0-4c6f-b1d5-189747d0cd79 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:28.649422 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0a1f18d7-8cb0-4c6f-b1d5-189747d0cd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:28.658884 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:28.766196 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3437a32-cce6-480b-ad66-6a8553abb553 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:28.779240 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 144/303] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:28.824445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e523f59-8282-44ed-b0ca-e164b946d1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:28.825699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e523f59-8282-44ed-b0ca-e164b946d1c8 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:28.825975 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e523f59-8282-44ed-b0ca-e164b946d1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:28.826246 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e523f59-8282-44ed-b0ca-e164b946d1c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:28.826939 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e523f59-8282-44ed-b0ca-e164b946d1c8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:28.827239 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e523f59-8282-44ed-b0ca-e164b946d1c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:28.828649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e523f59-8282-44ed-b0ca-e164b946d1c8 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:28.839114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 160/304] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:28.845250 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f18d7-8cb0-4c6f-b1d5-189747d0cd79 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:28.882044 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:28.988640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f18d7-8cb0-4c6f-b1d5-189747d0cd79 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:29.051978 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81cb74-a4ad-404a-bda1-4544746e597d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:29.053273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81cb74-a4ad-404a-bda1-4544746e597d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:29.053885 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81cb74-a4ad-404a-bda1-4544746e597d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:29.054573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81cb74-a4ad-404a-bda1-4544746e597d None admin] PATH_INFO: `/v3/users/cf17484ab8464f7bb8cea72eee190a24` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:29.075917 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81cb74-a4ad-404a-bda1-4544746e597d None admin] RBAC: Authorizing `identity:get_user(user_id=cf17484ab8464f7bb8cea72eee190a24)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:29.077888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81cb74-a4ad-404a-bda1-4544746e597d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:29.079953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 145/305] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:28 2022] GET /identity/v3/users/cf17484ab8464f7bb8cea72eee190a24 => generated 293 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:29.085627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e523f59-8282-44ed-b0ca-e164b946d1c8 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:29.127143 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81cb74-a4ad-404a-bda1-4544746e597d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:29.189447 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:29.199430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e523f59-8282-44ed-b0ca-e164b946d1c8 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:29.231904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:29.242450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1bb47-40e7-43c4-8cc4-dba1ca10a5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:29.243344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1bb47-40e7-43c4-8cc4-dba1ca10a5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:29.243623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1bb47-40e7-43c4-8cc4-dba1ca10a5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:29.243893 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1bb47-40e7-43c4-8cc4-dba1ca10a5f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:29.244633 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1bb47-40e7-43c4-8cc4-dba1ca10a5f8 None admin] RBAC: Adding query filter params (role.id=27a25b7936b8463c9e6d34d2cb49e61f, user.id=cf17484ab8464f7bb8cea72eee190a24, scope.system=all) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:29.245105 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1bb47-40e7-43c4-8cc4-dba1ca10a5f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:29.246731 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1bb47-40e7-43c4-8cc4-dba1ca10a5f8 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:29.249699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81cb74-a4ad-404a-bda1-4544746e597d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:29.252747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 161/306] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Sep 26 05:29:29 2022] GET /identity/v3/role_assignments?user.id=cf17484ab8464f7bb8cea72eee190a24&scope.system=all&role.id=27a25b7936b8463c9e6d34d2cb49e61f => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:29.294430 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa26f3-7b7e-4ca7-96e6-b24d7a96c59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:29.295543 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa26f3-7b7e-4ca7-96e6-b24d7a96c59b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:29.296130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa26f3-7b7e-4ca7-96e6-b24d7a96c59b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:29.296601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa26f3-7b7e-4ca7-96e6-b24d7a96c59b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:29.310143 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa26f3-7b7e-4ca7-96e6-b24d7a96c59b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:29.315863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-a6aa26f3-7b7e-4ca7-96e6-b24d7a96c59b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:29.315863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:29.315863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:29.315863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:29.315863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:29.315863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:29.315863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:29.315863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:29.315863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:29.315863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:29.315863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:29.315863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:29.315863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:29.315863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:29.315863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:29.315863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:29.316888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:29.316888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:29.316888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:29.316888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:29.316888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:29.316888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:29.316888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:29.316888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:29.316888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:29.316888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:29.316888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:29.316888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:29.316888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:29.316888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:29.316888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:29.316888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:29.317722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:29.317722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:29.317722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:29.317722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:29.317722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:29.317722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:29.317722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:29.317722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:29.317722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:29.317722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:29.317722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:29.317722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:29.320816 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 146/307] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:29.330423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1bb47-40e7-43c4-8cc4-dba1ca10a5f8 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:29.452566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1bb47-40e7-43c4-8cc4-dba1ca10a5f8 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:29.491923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6abf1-c74b-4ba4-b542-79e746c4e00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:29.492549 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6abf1-c74b-4ba4-b542-79e746c4e00f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:29.492826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6abf1-c74b-4ba4-b542-79e746c4e00f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:29.493095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6abf1-c74b-4ba4-b542-79e746c4e00f None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:29.493806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6abf1-c74b-4ba4-b542-79e746c4e00f None admin] RBAC: Adding query filter params (name=member) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:29.494107 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6abf1-c74b-4ba4-b542-79e746c4e00f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:29.495510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6abf1-c74b-4ba4-b542-79e746c4e00f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:29.505336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 162/308] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:29.511188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa26f3-7b7e-4ca7-96e6-b24d7a96c59b None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:29.628765 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa26f3-7b7e-4ca7-96e6-b24d7a96c59b None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:29.666716 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eacbee-ea7c-4952-869a-dd9eb845f92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:29.667767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eacbee-ea7c-4952-869a-dd9eb845f92d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:29.668370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eacbee-ea7c-4952-869a-dd9eb845f92d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:29.668910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eacbee-ea7c-4952-869a-dd9eb845f92d None admin] PATH_INFO: `/v3/users/f411494b67af4bf0a06f2e1bfed51474` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:29.670844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eacbee-ea7c-4952-869a-dd9eb845f92d None admin] RBAC: Authorizing `identity:get_user(user_id=f411494b67af4bf0a06f2e1bfed51474)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:29.672612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eacbee-ea7c-4952-869a-dd9eb845f92d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:29.674828 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 147/309] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:29 2022] GET /identity/v3/users/f411494b67af4bf0a06f2e1bfed51474 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:29.680375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6abf1-c74b-4ba4-b542-79e746c4e00f None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:29.702445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:29.797042 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6abf1-c74b-4ba4-b542-79e746c4e00f None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:29.835074 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc496f74-cb6d-4c05-b504-8c210679c105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:29.836118 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc496f74-cb6d-4c05-b504-8c210679c105 None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:29.836630 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc496f74-cb6d-4c05-b504-8c210679c105 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:29.837169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc496f74-cb6d-4c05-b504-8c210679c105 None admin] PATH_INFO: `/v3/system/users/f411494b67af4bf0a06f2e1bfed51474/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:29.839412 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc496f74-cb6d-4c05-b504-8c210679c105 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f411494b67af4bf0a06f2e1bfed51474, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:29.841125 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc496f74-cb6d-4c05-b504-8c210679c105 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:29.853771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 163/310] 10.222.0.57 () {58 vars in 1334 bytes} [Mon Sep 26 05:29:29 2022] PUT /identity/v3/system/users/f411494b67af4bf0a06f2e1bfed51474/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:30.293452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a21f3c-06df-4796-bcd6-20a2d233406a None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:30.294143 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a21f3c-06df-4796-bcd6-20a2d233406a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:30.294143 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a21f3c-06df-4796-bcd6-20a2d233406a None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:30.295760 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 148/311] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:30.301978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f21b8-2c04-4105-87d8-311b12e8a0ba None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:30.302747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f21b8-2c04-4105-87d8-311b12e8a0ba None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:30.302747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f21b8-2c04-4105-87d8-311b12e8a0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:30.329811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-839f21b8-2c04-4105-87d8-311b12e8a0ba None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:30.331812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-839f21b8-2c04-4105-87d8-311b12e8a0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:30.460159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 164/312] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:30.729795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a21f3c-06df-4796-bcd6-20a2d233406a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:30.750475 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:30.853646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a21f3c-06df-4796-bcd6-20a2d233406a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:30.892485 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4753c-5ec9-4922-a30a-6a1e1f6a4a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:30.893162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4753c-5ec9-4922-a30a-6a1e1f6a4a78 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:30.893439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4753c-5ec9-4922-a30a-6a1e1f6a4a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:30.893688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4753c-5ec9-4922-a30a-6a1e1f6a4a78 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:30.905558 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4753c-5ec9-4922-a30a-6a1e1f6a4a78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:30.910498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-e6a4753c-5ec9-4922-a30a-6a1e1f6a4a78 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:29:30.910498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:30.910498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:30.910498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:30.910498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:30.910498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:30.910498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:30.910498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:30.910498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:30.910498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:30.910498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:30.910498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:30.910498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:30.910498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:30.910498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:30.910498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:30.913752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:30.913752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:30.913752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:30.913752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:30.913752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:30.913752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:30.913752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:30.913752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:30.913752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:30.913752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:30.913752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:30.913752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:30.913752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:30.913752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:30.913752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:30.913752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:30.921557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:30.921557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:30.921557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:30.921557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:30.921557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:30.921557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:30.921557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:30.921557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:30.921557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:30.921557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:30.921557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:29:30.921557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:30.921557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 149/313] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:30.923457 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f21b8-2c04-4105-87d8-311b12e8a0ba None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:30.933862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:31.055544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f21b8-2c04-4105-87d8-311b12e8a0ba None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:31.094411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc660aec-5cbd-483b-af50-55625257d142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:31.095039 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc660aec-5cbd-483b-af50-55625257d142 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:31.095312 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc660aec-5cbd-483b-af50-55625257d142 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:31.095583 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc660aec-5cbd-483b-af50-55625257d142 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:31.096264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc660aec-5cbd-483b-af50-55625257d142 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:31.096560 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc660aec-5cbd-483b-af50-55625257d142 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:31.098865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc660aec-5cbd-483b-af50-55625257d142 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:31.115201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 165/314] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:31.120725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4753c-5ec9-4922-a30a-6a1e1f6a4a78 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:31.219105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:31.244230 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4753c-5ec9-4922-a30a-6a1e1f6a4a78 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:31.284468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba4d95-e58c-4eaa-8827-cab076eb0d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:31.285128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba4d95-e58c-4eaa-8827-cab076eb0d76 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:31.285443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba4d95-e58c-4eaa-8827-cab076eb0d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:31.285994 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba4d95-e58c-4eaa-8827-cab076eb0d76 None admin] PATH_INFO: `/v3/users/2666f4d720464cf2a8e3a64941933bb4` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:31.287255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba4d95-e58c-4eaa-8827-cab076eb0d76 None admin] RBAC: Authorizing `identity:get_user(user_id=2666f4d720464cf2a8e3a64941933bb4)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:31.288746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba4d95-e58c-4eaa-8827-cab076eb0d76 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:31.290893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 150/315] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:31 2022] GET /identity/v3/users/2666f4d720464cf2a8e3a64941933bb4 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:31.296717 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc660aec-5cbd-483b-af50-55625257d142 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:31.384419 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:31.423608 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc660aec-5cbd-483b-af50-55625257d142 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:31.427024 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26852fdb-b17e-471d-b990-d815e98f63b6 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:31.427534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26852fdb-b17e-471d-b990-d815e98f63b6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:31.428680 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26852fdb-b17e-471d-b990-d815e98f63b6 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:31.431693 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 151/316] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:29:31.437488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a151f-2263-46c4-b6b4-c43f116af7da None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:31.437794 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a151f-2263-46c4-b6b4-c43f116af7da None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:31.438069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a151f-2263-46c4-b6b4-c43f116af7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:31.461858 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7b0a151f-2263-46c4-b6b4-c43f116af7da None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:31.463568 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7b0a151f-2263-46c4-b6b4-c43f116af7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:31.469013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbe04e-01ce-4a36-af81-dfeadb615a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:31.470063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbe04e-01ce-4a36-af81-dfeadb615a61 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:31.470610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbe04e-01ce-4a36-af81-dfeadb615a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:31.471102 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbe04e-01ce-4a36-af81-dfeadb615a61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:31.565314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 152/317] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:29:31.574807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbe04e-01ce-4a36-af81-dfeadb615a61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:31.576639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbe04e-01ce-4a36-af81-dfeadb615a61 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:31.660928 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:31.692770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 166/318] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:31.698844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a151f-2263-46c4-b6b4-c43f116af7da None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:31.785161 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:31.814860 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a151f-2263-46c4-b6b4-c43f116af7da None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:31.855993 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367c9c3-b3a3-42e4-bcc0-85781ea588f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:31.857162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367c9c3-b3a3-42e4-bcc0-85781ea588f9 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:31.857655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367c9c3-b3a3-42e4-bcc0-85781ea588f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:31.858130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367c9c3-b3a3-42e4-bcc0-85781ea588f9 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:31.859615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367c9c3-b3a3-42e4-bcc0-85781ea588f9 None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:31.861416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367c9c3-b3a3-42e4-bcc0-85781ea588f9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:31.863775 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 153/319] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:31 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:31.870858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbe04e-01ce-4a36-af81-dfeadb615a61 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:31.999165 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbe04e-01ce-4a36-af81-dfeadb615a61 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:32.038854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367c9c3-b3a3-42e4-bcc0-85781ea588f9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:32.052718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdec5a1-ef4b-4dcc-94c2-b7fadc310299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:32.054251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdec5a1-ef4b-4dcc-94c2-b7fadc310299 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:32.055016 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdec5a1-ef4b-4dcc-94c2-b7fadc310299 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:32.055761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdec5a1-ef4b-4dcc-94c2-b7fadc310299 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:32.057339 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdec5a1-ef4b-4dcc-94c2-b7fadc310299 None admin] RBAC: Adding query filter params (role.id=27a25b7936b8463c9e6d34d2cb49e61f, scope.project.id=b13122687f1640669b462d728fc006ce, user.id=2666f4d720464cf2a8e3a64941933bb4) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:32.058159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdec5a1-ef4b-4dcc-94c2-b7fadc310299 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:32.061312 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdec5a1-ef4b-4dcc-94c2-b7fadc310299 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:32.073058 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 167/320] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:29:31 2022] GET /identity/v3/role_assignments?user.id=2666f4d720464cf2a8e3a64941933bb4&scope.project.id=b13122687f1640669b462d728fc006ce&role.id=27a25b7936b8463c9e6d34d2cb49e61f => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:32.151097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1b216-de1e-4573-806a-49e2cae5bcea None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:32.151378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1b216-de1e-4573-806a-49e2cae5bcea None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:32.151813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1b216-de1e-4573-806a-49e2cae5bcea None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:32.153291 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 168/321] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:29:32.159311 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90802092-da1d-4730-b135-0106d1bfb4b8 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:32.159566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90802092-da1d-4730-b135-0106d1bfb4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:32.159880 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90802092-da1d-4730-b135-0106d1bfb4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:32.175393 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367c9c3-b3a3-42e4-bcc0-85781ea588f9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:32.181991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-90802092-da1d-4730-b135-0106d1bfb4b8 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:32.183328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-90802092-da1d-4730-b135-0106d1bfb4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:32.224126 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90d184-9049-44e9-8d58-c98f69e2550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:32.224758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90d184-9049-44e9-8d58-c98f69e2550e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:32.225042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90d184-9049-44e9-8d58-c98f69e2550e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:32.225395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90d184-9049-44e9-8d58-c98f69e2550e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:32.226749 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90d184-9049-44e9-8d58-c98f69e2550e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:32.235770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90d184-9049-44e9-8d58-c98f69e2550e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:32.235770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 154/322] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:29:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:32.245473 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90d184-9049-44e9-8d58-c98f69e2550e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:32.258266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:32.262149 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:32.336916 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 169/323] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:29:32.398938 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90d184-9049-44e9-8d58-c98f69e2550e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:32.451011 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c639fa3-a777-40d6-8fe2-f5cbe0083dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:32.452489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c639fa3-a777-40d6-8fe2-f5cbe0083dfa None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:32.452966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c639fa3-a777-40d6-8fe2-f5cbe0083dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:32.453470 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c639fa3-a777-40d6-8fe2-f5cbe0083dfa None admin] PATH_INFO: `/v3/groups` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:32.454342 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c639fa3-a777-40d6-8fe2-f5cbe0083dfa None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:32.456040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c639fa3-a777-40d6-8fe2-f5cbe0083dfa None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:32.475734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 155/324] 10.222.0.57 () {60 vars in 1134 bytes} [Mon Sep 26 05:29:32 2022] POST /identity/v3/groups => generated 226 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:29:32.622689 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90802092-da1d-4730-b135-0106d1bfb4b8 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:32.794174 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90802092-da1d-4730-b135-0106d1bfb4b8 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:32.806890 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5f708-b3d7-4102-b2ab-430a9355f331 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:32.807377 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5f708-b3d7-4102-b2ab-430a9355f331 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:32.807861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5f708-b3d7-4102-b2ab-430a9355f331 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:32.810539 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 156/325] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:32.819636 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66648c4b-ccdd-4383-9460-f601dc95ba7d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:32.820142 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66648c4b-ccdd-4383-9460-f601dc95ba7d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:32.820601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66648c4b-ccdd-4383-9460-f601dc95ba7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:32.846201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e4dd3-a7ba-47c8-847f-b117b8d805f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:32.846840 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e4dd3-a7ba-47c8-847f-b117b8d805f8 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:32.847114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e4dd3-a7ba-47c8-847f-b117b8d805f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:32.847727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e4dd3-a7ba-47c8-847f-b117b8d805f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:32.859820 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-66648c4b-ccdd-4383-9460-f601dc95ba7d None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:32.861214 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e4dd3-a7ba-47c8-847f-b117b8d805f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:32.861977 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-66648c4b-ccdd-4383-9460-f601dc95ba7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:32.866431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-687e4dd3-a7ba-47c8-847f-b117b8d805f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:32.866431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:32.866431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:32.866431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:32.866431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:32.866431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:32.866431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:32.866431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:32.866431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:32.866431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:32.866431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:32.866431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:32.866431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:32.866431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:32.866431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:32.866431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:32.868152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:32.868152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:32.868152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:32.868152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:32.868152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:32.868152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:32.868152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:32.868152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:32.868152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:32.868152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:32.868152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:32.868152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:32.868152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:32.868152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:32.868152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:32.868152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:32.869883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:32.869883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:32.869883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:32.869883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:32.869883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:32.869883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:32.869883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:32.869883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:32.869883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:32.869883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:32.869883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:32.869883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:32.869883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 170/326] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:32.879144 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e4dd3-a7ba-47c8-847f-b117b8d805f8 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:32.921795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:32.973793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:32.981073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 157/327] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:33.042379 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e4dd3-a7ba-47c8-847f-b117b8d805f8 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:33.084894 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051bba6-e12b-420e-94ba-2b4b5a8bd908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:33.085949 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051bba6-e12b-420e-94ba-2b4b5a8bd908 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:33.086420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051bba6-e12b-420e-94ba-2b4b5a8bd908 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:33.086920 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051bba6-e12b-420e-94ba-2b4b5a8bd908 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:33.087903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051bba6-e12b-420e-94ba-2b4b5a8bd908 None admin] RBAC: Adding query filter params (name=member) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:33.088421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051bba6-e12b-420e-94ba-2b4b5a8bd908 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:33.090550 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051bba6-e12b-420e-94ba-2b4b5a8bd908 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:33.101915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 171/328] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:33.110690 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66648c4b-ccdd-4383-9460-f601dc95ba7d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:33.240730 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66648c4b-ccdd-4383-9460-f601dc95ba7d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:33.287456 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871484e0-40d5-4bb9-a87d-1d7eab97e604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:33.288138 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871484e0-40d5-4bb9-a87d-1d7eab97e604 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:33.288411 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871484e0-40d5-4bb9-a87d-1d7eab97e604 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:33.288667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871484e0-40d5-4bb9-a87d-1d7eab97e604 None admin] PATH_INFO: `/v3/users/f411494b67af4bf0a06f2e1bfed51474` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:33.292210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871484e0-40d5-4bb9-a87d-1d7eab97e604 None admin] RBAC: Authorizing `identity:get_user(user_id=f411494b67af4bf0a06f2e1bfed51474)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:33.293884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871484e0-40d5-4bb9-a87d-1d7eab97e604 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:33.296000 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 158/329] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:33 2022] GET /identity/v3/users/f411494b67af4bf0a06f2e1bfed51474 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:33.305525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051bba6-e12b-420e-94ba-2b4b5a8bd908 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:33.502238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871484e0-40d5-4bb9-a87d-1d7eab97e604 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:33.519802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:33.535956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051bba6-e12b-420e-94ba-2b4b5a8bd908 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:33.608084 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8635bcb-6803-4ca8-8113-4614b9209cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:33.610119 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8635bcb-6803-4ca8-8113-4614b9209cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:33.610940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8635bcb-6803-4ca8-8113-4614b9209cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:33.611696 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8635bcb-6803-4ca8-8113-4614b9209cc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:33.613442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8635bcb-6803-4ca8-8113-4614b9209cc4 None admin] RBAC: Adding query filter params (role.id=bb454faf5dd643b892c6f9b670ad1f2d, user.id=f411494b67af4bf0a06f2e1bfed51474, scope.system=all) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:33.614236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8635bcb-6803-4ca8-8113-4614b9209cc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:33.617423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8635bcb-6803-4ca8-8113-4614b9209cc4 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:33.634357 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:33.642331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 172/330] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Sep 26 05:29:33 2022] GET /identity/v3/role_assignments?user.id=f411494b67af4bf0a06f2e1bfed51474&scope.system=all&role.id=bb454faf5dd643b892c6f9b670ad1f2d => generated 510 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:33.682028 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871484e0-40d5-4bb9-a87d-1d7eab97e604 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:33.732801 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c4b04-ed46-4fe4-bf24-f1660d0af849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:33.733489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c4b04-ed46-4fe4-bf24-f1660d0af849 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:33.733758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c4b04-ed46-4fe4-bf24-f1660d0af849 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:33.734037 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c4b04-ed46-4fe4-bf24-f1660d0af849 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:33.751844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c4b04-ed46-4fe4-bf24-f1660d0af849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:33.757163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-f50c4b04-ed46-4fe4-bf24-f1660d0af849 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:29:33.757163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:33.757163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:33.757163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:33.757163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:33.757163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:33.757163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:33.757163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:33.757163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:33.757163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:33.757163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:33.757163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:33.757163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:33.757163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:33.757163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:33.757163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:33.758874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:33.758874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:33.758874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:33.758874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:33.758874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:33.758874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:33.758874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:33.758874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:33.758874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:33.758874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:33.758874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:33.758874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:33.758874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:33.758874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:33.758874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:33.758874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:33.760460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:33.760460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:33.760460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:33.760460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:33.760460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:33.760460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:33.760460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:33.760460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:33.760460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:33.760460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:33.760460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:29:33.760460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:33.760460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 159/331] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:33.767984 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8635bcb-6803-4ca8-8113-4614b9209cc4 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:34.046901 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8635bcb-6803-4ca8-8113-4614b9209cc4 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:34.137222 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e72e7-f817-4cce-9a5b-046543b8752f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:34.138253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e72e7-f817-4cce-9a5b-046543b8752f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:34.138736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e72e7-f817-4cce-9a5b-046543b8752f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:34.139213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e72e7-f817-4cce-9a5b-046543b8752f None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:34.140448 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e72e7-f817-4cce-9a5b-046543b8752f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:34.140985 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e72e7-f817-4cce-9a5b-046543b8752f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:34.143640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e72e7-f817-4cce-9a5b-046543b8752f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:34.164562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 173/332] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:34.173079 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c4b04-ed46-4fe4-bf24-f1660d0af849 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:34.362382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c4b04-ed46-4fe4-bf24-f1660d0af849 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:34.415424 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:34.430055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad749a-6ac6-436e-a377-51ac15067b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:34.431711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad749a-6ac6-436e-a377-51ac15067b6f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:34.432499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad749a-6ac6-436e-a377-51ac15067b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:34.433526 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad749a-6ac6-436e-a377-51ac15067b6f None admin] PATH_INFO: `/v3/users/cf17484ab8464f7bb8cea72eee190a24` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:34.435673 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad749a-6ac6-436e-a377-51ac15067b6f None admin] RBAC: Authorizing `identity:get_user(user_id=cf17484ab8464f7bb8cea72eee190a24)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:34.438194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad749a-6ac6-436e-a377-51ac15067b6f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:34.441577 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 160/333] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:34 2022] GET /identity/v3/users/cf17484ab8464f7bb8cea72eee190a24 => generated 293 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:34.448191 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e72e7-f817-4cce-9a5b-046543b8752f None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:34.475447 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:34.622198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e72e7-f817-4cce-9a5b-046543b8752f None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:34.690172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7040a4-a8b7-420a-bae4-eaa8b5325016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:34.691286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7040a4-a8b7-420a-bae4-eaa8b5325016 None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:34.691764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7040a4-a8b7-420a-bae4-eaa8b5325016 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:34.692220 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7040a4-a8b7-420a-bae4-eaa8b5325016 None admin] PATH_INFO: `/v3/system/users/cf17484ab8464f7bb8cea72eee190a24/roles/27a25b7936b8463c9e6d34d2cb49e61f` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:34.698603 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7040a4-a8b7-420a-bae4-eaa8b5325016 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cf17484ab8464f7bb8cea72eee190a24, role_id=27a25b7936b8463c9e6d34d2cb49e61f)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:34.701316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7040a4-a8b7-420a-bae4-eaa8b5325016 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:34.719509 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 174/334] 10.222.0.57 () {58 vars in 1334 bytes} [Mon Sep 26 05:29:34 2022] PUT /identity/v3/system/users/cf17484ab8464f7bb8cea72eee190a24/roles/27a25b7936b8463c9e6d34d2cb49e61f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:37.024901 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc0f2b-71f8-4864-ac70-ce468ef5f68d None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:37.025999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc0f2b-71f8-4864-ac70-ce468ef5f68d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:37.026489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc0f2b-71f8-4864-ac70-ce468ef5f68d None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:37.028652 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 161/335] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:37.038366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505f537-9e9c-4ca4-9ad7-ed5f89be537f None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:37.038947 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505f537-9e9c-4ca4-9ad7-ed5f89be537f None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:37.039494 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505f537-9e9c-4ca4-9ad7-ed5f89be537f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:37.081836 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-c505f537-9e9c-4ca4-9ad7-ed5f89be537f None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:37.083978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c505f537-9e9c-4ca4-9ad7-ed5f89be537f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:37.249378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 175/336] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:37.335302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bcd51-98ba-4f44-9a24-e8857833e4e9 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:37.335790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bcd51-98ba-4f44-9a24-e8857833e4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:37.336272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bcd51-98ba-4f44-9a24-e8857833e4e9 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:37.338786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 162/337] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:37.349886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0390f1-2d9c-489c-9877-ad450bc12e68 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:37.350364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0390f1-2d9c-489c-9877-ad450bc12e68 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:37.350822 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0390f1-2d9c-489c-9877-ad450bc12e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:37.363380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:37.383929 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-9c0390f1-2d9c-489c-9877-ad450bc12e68 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:37.386285 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9c0390f1-2d9c-489c-9877-ad450bc12e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:37.542599 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 176/338] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:29:37.940275 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa4f4b-6099-48de-a201-6348401f67c6 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:37.941115 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa4f4b-6099-48de-a201-6348401f67c6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:37.941891 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa4f4b-6099-48de-a201-6348401f67c6 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:37.944583 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 163/339] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:37.951418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cf9b8-6c78-467c-a922-705a6576c5b7 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:37.952313 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cf9b8-6c78-467c-a922-705a6576c5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:37.953080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cf9b8-6c78-467c-a922-705a6576c5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:37.990491 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-718cf9b8-6c78-467c-a922-705a6576c5b7 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:37.992967 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-718cf9b8-6c78-467c-a922-705a6576c5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:38.056182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa4f4b-6099-48de-a201-6348401f67c6 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:38.224311 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 177/340] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:38.369566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cf9b8-6c78-467c-a922-705a6576c5b7 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:38.377170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa4f4b-6099-48de-a201-6348401f67c6 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:38.471492 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed4aa6-0cd2-4b4a-a0c7-f5650a8e3c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:38.473521 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed4aa6-0cd2-4b4a-a0c7-f5650a8e3c5b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:38.474297 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed4aa6-0cd2-4b4a-a0c7-f5650a8e3c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:38.475140 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed4aa6-0cd2-4b4a-a0c7-f5650a8e3c5b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:38.510721 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed4aa6-0cd2-4b4a-a0c7-f5650a8e3c5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:38.526338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-69ed4aa6-0cd2-4b4a-a0c7-f5650a8e3c5b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:38.526338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:38.526338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:38.526338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:38.526338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:38.526338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:38.526338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:38.526338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:38.526338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:38.526338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:38.526338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:38.526338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:38.526338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:38.526338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:38.526338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:38.526338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:38.528139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:38.528139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:38.528139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:38.528139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:38.528139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:38.528139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:38.528139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:38.528139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:38.528139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:38.528139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:38.528139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:38.528139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:38.528139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:38.528139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:38.528139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:38.528139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:38.529791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:38.529791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:38.529791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:38.529791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:38.529791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:38.529791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:38.529791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:38.529791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:38.529791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:38.529791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:38.529791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:38.529791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:38.534116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 164/341] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:29:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 479 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:38.543710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a5542-4b4a-4902-9d44-188326b22320 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:38.544531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a5542-4b4a-4902-9d44-188326b22320 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:38.545330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a5542-4b4a-4902-9d44-188326b22320 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:38.546864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:38.551766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 165/342] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:38 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:38.554008 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a5542-4b4a-4902-9d44-188326b22320 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:38.636001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cf9b8-6c78-467c-a922-705a6576c5b7 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:38.700086 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:38.700341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03083fe-f721-47d0-9c1d-65e8d3953941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:38.701644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03083fe-f721-47d0-9c1d-65e8d3953941 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:38.701928 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03083fe-f721-47d0-9c1d-65e8d3953941 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:38.702204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03083fe-f721-47d0-9c1d-65e8d3953941 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:38.717853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03083fe-f721-47d0-9c1d-65e8d3953941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:38.723345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-f03083fe-f721-47d0-9c1d-65e8d3953941 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:38.723345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:38.723345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:38.723345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:38.723345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:38.723345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:38.723345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:38.723345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:38.723345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:38.723345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:38.723345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:38.723345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:38.723345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:38.723345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:38.723345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:38.723345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:38.725003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:38.725003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:38.725003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:38.725003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:38.725003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:38.725003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:38.725003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:38.725003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:38.725003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:38.725003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:38.725003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:38.725003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:38.725003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:38.725003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:38.725003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:38.725003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:38.727612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:38.727612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:38.727612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:38.727612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:38.727612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:38.727612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:38.727612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:38.727612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:38.727612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:38.727612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:38.727612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:38.727612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:38.727612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 178/343] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:29:38.727612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c046a96-2dff-4abb-8f56-2b4b6058601b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:38.732020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c046a96-2dff-4abb-8f56-2b4b6058601b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:38.732020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c046a96-2dff-4abb-8f56-2b4b6058601b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:38.750308 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-9c046a96-2dff-4abb-8f56-2b4b6058601b None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:38.751537 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9c046a96-2dff-4abb-8f56-2b4b6058601b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:38.816936 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a5542-4b4a-4902-9d44-188326b22320 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:38.838611 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 179/344] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:38.839552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c046a96-2dff-4abb-8f56-2b4b6058601b None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:38.904947 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cceae-fa44-49d6-b859-a93152ff4007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:38.906586 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cceae-fa44-49d6-b859-a93152ff4007 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:38.909635 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cceae-fa44-49d6-b859-a93152ff4007 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:38.912419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cceae-fa44-49d6-b859-a93152ff4007 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:38.914092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cceae-fa44-49d6-b859-a93152ff4007 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:38.915204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cceae-fa44-49d6-b859-a93152ff4007 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:38.921218 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cceae-fa44-49d6-b859-a93152ff4007 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:38.948160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 166/345] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:29:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:38.955461 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cceae-fa44-49d6-b859-a93152ff4007 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:39.024643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:39.140360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c046a96-2dff-4abb-8f56-2b4b6058601b None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:39.197034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cceae-fa44-49d6-b859-a93152ff4007 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:39.204020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730576f-14b6-464e-b2d8-eb33d9fa67f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:39.204494 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730576f-14b6-464e-b2d8-eb33d9fa67f4 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:39.204751 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730576f-14b6-464e-b2d8-eb33d9fa67f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:39.205068 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730576f-14b6-464e-b2d8-eb33d9fa67f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:39.218141 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730576f-14b6-464e-b2d8-eb33d9fa67f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:39.224173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-2730576f-14b6-464e-b2d8-eb33d9fa67f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:39.224173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:39.224173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:39.224173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:39.224173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:39.224173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:39.224173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:39.224173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:39.224173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:39.224173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:39.224173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:39.224173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:39.224173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:39.224173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:39.224173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:39.224173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:39.226813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:39.226813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:39.226813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:39.226813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:39.226813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:39.226813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:39.226813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:39.226813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:39.226813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:39.226813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:39.226813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:39.226813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:39.226813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:39.226813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:39.226813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:39.226813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:39.228381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:39.228381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:39.228381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:39.228381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:39.228381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:39.228381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:39.228381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:39.228381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:39.228381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:39.228381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:39.228381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:39.228381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:39.228381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 180/346] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:39.228381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730576f-14b6-464e-b2d8-eb33d9fa67f4 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:39.259369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:39.292208 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9cefb-8a88-4205-ae6f-d4a826c19d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:39.293478 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9cefb-8a88-4205-ae6f-d4a826c19d52 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:39.294078 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9cefb-8a88-4205-ae6f-d4a826c19d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:39.294644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9cefb-8a88-4205-ae6f-d4a826c19d52 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:39.296489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9cefb-8a88-4205-ae6f-d4a826c19d52 None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:39.296770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9cefb-8a88-4205-ae6f-d4a826c19d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:39.299745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9cefb-8a88-4205-ae6f-d4a826c19d52 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:39.315844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 167/347] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:39.318089 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9cefb-8a88-4205-ae6f-d4a826c19d52 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:39.388517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730576f-14b6-464e-b2d8-eb33d9fa67f4 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:39.415176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:39.453944 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6634e4-44fa-452d-90be-f449c88ba9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:39.455552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6634e4-44fa-452d-90be-f449c88ba9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:39.455552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6634e4-44fa-452d-90be-f449c88ba9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:39.455552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6634e4-44fa-452d-90be-f449c88ba9d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:39.497591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9cefb-8a88-4205-ae6f-d4a826c19d52 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:39.545268 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa8c7d-daa4-4ad3-b63a-330cdf6c4490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:39.546193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa8c7d-daa4-4ad3-b63a-330cdf6c4490 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:39.546461 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa8c7d-daa4-4ad3-b63a-330cdf6c4490 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:39.546739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa8c7d-daa4-4ad3-b63a-330cdf6c4490 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:39.563040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa8c7d-daa4-4ad3-b63a-330cdf6c4490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:39.570549 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6634e4-44fa-452d-90be-f449c88ba9d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:39.573029 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6634e4-44fa-452d-90be-f449c88ba9d4 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:39.573203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-bafa8c7d-daa4-4ad3-b63a-330cdf6c4490 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:39.573203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:39.573203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:39.573203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:39.573203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:39.573203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:39.573203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:39.573203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:39.573203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:39.573203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:39.573203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:39.573203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:39.573203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:39.573203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:39.573203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:39.573203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:39.574319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:39.574319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:39.574319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:39.574319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:39.574319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:39.574319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:39.574319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:39.574319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:39.574319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:39.574319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:39.574319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:39.574319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:39.574319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:39.574319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:39.574319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:39.574319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:39.575269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:39.575269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:39.575269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:39.575269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:39.575269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:39.575269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:39.575269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:39.575269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:39.575269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:39.575269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:39.575269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:39.575269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:39.575269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 168/348] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:29:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:39.576386 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa8c7d-daa4-4ad3-b63a-330cdf6c4490 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:39.639013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:39.727874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa8c7d-daa4-4ad3-b63a-330cdf6c4490 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:39.739973 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 181/349] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:39.741558 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c0cfd-df24-487b-ba41-5d819c8e5873 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:39.742041 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c0cfd-df24-487b-ba41-5d819c8e5873 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:39.742492 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c0cfd-df24-487b-ba41-5d819c8e5873 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:39.745004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 182/350] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:39.746508 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c0cfd-df24-487b-ba41-5d819c8e5873 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:39.774330 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:39.777591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d4188-617d-4c8f-a753-b124274895ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:39.777591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d4188-617d-4c8f-a753-b124274895ed None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:39.777591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d4188-617d-4c8f-a753-b124274895ed None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:39.778096 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d4188-617d-4c8f-a753-b124274895ed None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:39.779780 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d4188-617d-4c8f-a753-b124274895ed None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:39.779780 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d4188-617d-4c8f-a753-b124274895ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:39.782108 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d4188-617d-4c8f-a753-b124274895ed None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:39.796663 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 169/351] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:39.798316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafdda3-5a02-45cc-8aca-5c0b8249cfb6 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:39.798633 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafdda3-5a02-45cc-8aca-5c0b8249cfb6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:39.799092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aafdda3-5a02-45cc-8aca-5c0b8249cfb6 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:39.801874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 170/352] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:39.802717 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafdda3-5a02-45cc-8aca-5c0b8249cfb6 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:39.917620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c0cfd-df24-487b-ba41-5d819c8e5873 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:39.969235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af61333-7387-4402-afcc-992f66149c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:39.970176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af61333-7387-4402-afcc-992f66149c0d None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:39.970603 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af61333-7387-4402-afcc-992f66149c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:39.971062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af61333-7387-4402-afcc-992f66149c0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:39.985396 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aafdda3-5a02-45cc-8aca-5c0b8249cfb6 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:39.987351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:40.042175 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a83e5-afa5-4d03-82b9-978880fd0783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:40.043286 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a83e5-afa5-4d03-82b9-978880fd0783 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:40.043781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a83e5-afa5-4d03-82b9-978880fd0783 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:40.044059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a83e5-afa5-4d03-82b9-978880fd0783 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:40.044948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a83e5-afa5-4d03-82b9-978880fd0783 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:40.045063 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a83e5-afa5-4d03-82b9-978880fd0783 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:40.046505 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a83e5-afa5-4d03-82b9-978880fd0783 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:40.056509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 171/353] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:29:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:40.057410 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a83e5-afa5-4d03-82b9-978880fd0783 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:40.092384 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af61333-7387-4402-afcc-992f66149c0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:40.096111 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af61333-7387-4402-afcc-992f66149c0d None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:40.124911 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:40.231796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a83e5-afa5-4d03-82b9-978880fd0783 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:40.265040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 183/354] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:40.267021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95feee-4f2c-4496-995b-6f46da7a2169 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:40.267495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95feee-4f2c-4496-995b-6f46da7a2169 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:40.267930 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95feee-4f2c-4496-995b-6f46da7a2169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:40.288519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d116ecc-71d7-4aed-8b0a-a9678b04ad1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:40.289288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d116ecc-71d7-4aed-8b0a-a9678b04ad1c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:40.289529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d116ecc-71d7-4aed-8b0a-a9678b04ad1c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:40.289762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d116ecc-71d7-4aed-8b0a-a9678b04ad1c None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:40.291009 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d116ecc-71d7-4aed-8b0a-a9678b04ad1c None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:40.292467 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d116ecc-71d7-4aed-8b0a-a9678b04ad1c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:40.294283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 172/355] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:40 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:40.298492 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d116ecc-71d7-4aed-8b0a-a9678b04ad1c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:40.298948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-af95feee-4f2c-4496-995b-6f46da7a2169 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:40.301502 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-af95feee-4f2c-4496-995b-6f46da7a2169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:40.352141 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:40.427365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 184/356] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:40.430450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a04048-003d-42bf-b168-6c1c7141bfcd None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:40.430450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a04048-003d-42bf-b168-6c1c7141bfcd None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:40.430450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a04048-003d-42bf-b168-6c1c7141bfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:40.460978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-70a04048-003d-42bf-b168-6c1c7141bfcd None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:40.462527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-70a04048-003d-42bf-b168-6c1c7141bfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:40.472741 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d116ecc-71d7-4aed-8b0a-a9678b04ad1c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:40.519337 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2390da4-bcef-4e7b-bb65-e38efaf55b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:40.519979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2390da4-bcef-4e7b-bb65-e38efaf55b8e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:40.520258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2390da4-bcef-4e7b-bb65-e38efaf55b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:40.520531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2390da4-bcef-4e7b-bb65-e38efaf55b8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:40.561061 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:40.617551 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 185/357] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:40.618966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a04048-003d-42bf-b168-6c1c7141bfcd None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:40.651122 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2390da4-bcef-4e7b-bb65-e38efaf55b8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:40.654080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2390da4-bcef-4e7b-bb65-e38efaf55b8e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:40.717743 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:40.813743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a04048-003d-42bf-b168-6c1c7141bfcd None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:40.858338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 173/358] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:40.860092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2390da4-bcef-4e7b-bb65-e38efaf55b8e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:40.878696 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2f018-35e0-431b-9cc5-e420eba8ae96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:40.879604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2f018-35e0-431b-9cc5-e420eba8ae96 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:40.880076 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2f018-35e0-431b-9cc5-e420eba8ae96 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:40.880547 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2f018-35e0-431b-9cc5-e420eba8ae96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:40.957413 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:40.991007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2f018-35e0-431b-9cc5-e420eba8ae96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:40.992751 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2f018-35e0-431b-9cc5-e420eba8ae96 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:41.015612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2390da4-bcef-4e7b-bb65-e38efaf55b8e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:41.049039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:41.055036 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b47b3-aae9-43ac-9f9e-fa054e1fd60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:41.055685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b47b3-aae9-43ac-9f9e-fa054e1fd60f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:41.055963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b47b3-aae9-43ac-9f9e-fa054e1fd60f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:41.056234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b47b3-aae9-43ac-9f9e-fa054e1fd60f None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:41.057509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b47b3-aae9-43ac-9f9e-fa054e1fd60f None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:41.059070 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b47b3-aae9-43ac-9f9e-fa054e1fd60f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:41.060766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 174/359] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:40 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:41.061625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b47b3-aae9-43ac-9f9e-fa054e1fd60f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:41.115700 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 186/360] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:41.117048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2f018-35e0-431b-9cc5-e420eba8ae96 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:41.186247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b47b3-aae9-43ac-9f9e-fa054e1fd60f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:41.216969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:41.234340 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff1dd3-b71d-43a2-a74b-a9dbd4f91baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:41.235557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff1dd3-b71d-43a2-a74b-a9dbd4f91baf None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:41.235685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff1dd3-b71d-43a2-a74b-a9dbd4f91baf None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:41.235933 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff1dd3-b71d-43a2-a74b-a9dbd4f91baf None admin] PATH_INFO: `/v3/groups/8763f0cf0bf74aa8a2f27c494160c014` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:41.243270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff1dd3-b71d-43a2-a74b-a9dbd4f91baf None admin] RBAC: Authorizing `identity:get_group(group_id=8763f0cf0bf74aa8a2f27c494160c014)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:41.245076 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff1dd3-b71d-43a2-a74b-a9dbd4f91baf None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:41.247271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 175/361] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 26 05:29:41 2022] GET /identity/v3/groups/8763f0cf0bf74aa8a2f27c494160c014 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:41.248616 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff1dd3-b71d-43a2-a74b-a9dbd4f91baf None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:41.256751 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2f018-35e0-431b-9cc5-e420eba8ae96 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:41.303245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1c3f0-8496-4c88-818d-6d5359e4a36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:41.304209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1c3f0-8496-4c88-818d-6d5359e4a36d None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:41.304645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1c3f0-8496-4c88-818d-6d5359e4a36d None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:41.305075 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e1c3f0-8496-4c88-818d-6d5359e4a36d None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:41.306861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1c3f0-8496-4c88-818d-6d5359e4a36d None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:41.309305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e1c3f0-8496-4c88-818d-6d5359e4a36d None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:41.316460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 187/362] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:41 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:41.316460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1c3f0-8496-4c88-818d-6d5359e4a36d None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:41.340841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:41.386406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff1dd3-b71d-43a2-a74b-a9dbd4f91baf None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:41.425574 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c86c1e-ab01-4d62-981f-42d0c2336133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:41.426234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c86c1e-ab01-4d62-981f-42d0c2336133 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:41.426513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c86c1e-ab01-4d62-981f-42d0c2336133 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:41.426785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c86c1e-ab01-4d62-981f-42d0c2336133 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:41.440874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c86c1e-ab01-4d62-981f-42d0c2336133 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:41.441925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e1c3f0-8496-4c88-818d-6d5359e4a36d None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:41.451533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-f5c86c1e-ab01-4d62-981f-42d0c2336133 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:29:41.451533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:41.451533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:41.451533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:41.451533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:41.451533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:41.451533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:41.451533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:41.451533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:41.451533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:41.451533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:41.451533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:41.451533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:41.451533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:41.451533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:41.451533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:41.453499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:41.453499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:41.453499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:41.453499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:41.453499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:41.453499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:41.453499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:41.453499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:41.453499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:41.453499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:41.453499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:41.453499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:41.453499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:41.453499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:41.453499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:41.453499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:41.455001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:41.455001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:41.455001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:41.455001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:41.455001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:41.455001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:41.455001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:41.455001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:41.455001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:41.455001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:41.455001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 05:29:41.455001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:41.455001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 176/363] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:41.455686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c86c1e-ab01-4d62-981f-42d0c2336133 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:41.487313 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33daa84c-3ca0-4a6c-8e86-8110e94b9275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:41.487965 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33daa84c-3ca0-4a6c-8e86-8110e94b9275 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:41.488239 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33daa84c-3ca0-4a6c-8e86-8110e94b9275 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:41.488503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33daa84c-3ca0-4a6c-8e86-8110e94b9275 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:41.502269 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33daa84c-3ca0-4a6c-8e86-8110e94b9275 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:41.508092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-33daa84c-3ca0-4a6c-8e86-8110e94b9275 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:29:41.508092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:41.508092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:41.508092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:41.508092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:41.508092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:41.508092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:41.508092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:41.508092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:41.508092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:41.508092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:41.508092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:41.508092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:41.508092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:41.508092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:41.508092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:41.509803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:41.509803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:41.509803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:41.509803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:41.509803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:41.509803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:41.509803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:41.509803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:41.509803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:41.509803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:41.509803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:41.509803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:41.509803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:41.509803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:41.509803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:41.509803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:41.512146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:41.512146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:41.512146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:41.512146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:41.512146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:41.512146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:41.512146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:41.512146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:41.512146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:41.512146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:41.512146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:29:41.512146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:41.512146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 188/364] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:29:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:41.512146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33daa84c-3ca0-4a6c-8e86-8110e94b9275 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:41.516489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:41.610784 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c86c1e-ab01-4d62-981f-42d0c2336133 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:41.636029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:41.648352 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa086e79-52e8-4eb3-a0f4-f3cf53c890ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:41.648987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa086e79-52e8-4eb3-a0f4-f3cf53c890ba None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:41.649303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa086e79-52e8-4eb3-a0f4-f3cf53c890ba None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:41.649618 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa086e79-52e8-4eb3-a0f4-f3cf53c890ba None admin] PATH_INFO: `/v3/groups/8763f0cf0bf74aa8a2f27c494160c014` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:41.650798 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa086e79-52e8-4eb3-a0f4-f3cf53c890ba None admin] RBAC: Authorizing `identity:get_group(group_id=8763f0cf0bf74aa8a2f27c494160c014)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:41.652281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa086e79-52e8-4eb3-a0f4-f3cf53c890ba None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:41.653980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33daa84c-3ca0-4a6c-8e86-8110e94b9275 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:41.654156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 177/365] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 26 05:29:41 2022] GET /identity/v3/groups/8763f0cf0bf74aa8a2f27c494160c014 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:41.654807 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa086e79-52e8-4eb3-a0f4-f3cf53c890ba None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:41.704467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9671d-33cb-40a5-97f1-cebd21f38349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:41.705198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9671d-33cb-40a5-97f1-cebd21f38349 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:41.705662 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9671d-33cb-40a5-97f1-cebd21f38349 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:41.706062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9671d-33cb-40a5-97f1-cebd21f38349 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:41.707464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9671d-33cb-40a5-97f1-cebd21f38349 None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:41.709351 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9671d-33cb-40a5-97f1-cebd21f38349 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:41.711194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 189/366] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:41 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:41.712545 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9671d-33cb-40a5-97f1-cebd21f38349 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:41.784135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa086e79-52e8-4eb3-a0f4-f3cf53c890ba None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:41.803555 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:41.835095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7d32b-074e-4ffc-bdd4-9aecdb0f976e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:41.835987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7d32b-074e-4ffc-bdd4-9aecdb0f976e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:41.836341 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7d32b-074e-4ffc-bdd4-9aecdb0f976e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:41.836672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7d32b-074e-4ffc-bdd4-9aecdb0f976e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:41.837735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7d32b-074e-4ffc-bdd4-9aecdb0f976e None admin] RBAC: Adding query filter params (group.id=8763f0cf0bf74aa8a2f27c494160c014, role.id=05e78a4a009c41488cf3d2c47e8cbe30, scope.project.id=4e48d801558a426b8b994cb2bab00a20) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:41.838107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7d32b-074e-4ffc-bdd4-9aecdb0f976e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:41.840025 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7d32b-074e-4ffc-bdd4-9aecdb0f976e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:41.848027 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 178/367] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 26 05:29:41 2022] GET /identity/v3/role_assignments?group.id=8763f0cf0bf74aa8a2f27c494160c014&scope.project.id=4e48d801558a426b8b994cb2bab00a20&role.id=05e78a4a009c41488cf3d2c47e8cbe30 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:41.849302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7d32b-074e-4ffc-bdd4-9aecdb0f976e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:41.849637 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9671d-33cb-40a5-97f1-cebd21f38349 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:41.906732 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001c2fe-3828-4eea-91c3-1f6e0401aa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:41.907570 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001c2fe-3828-4eea-91c3-1f6e0401aa7e None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:41.908045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001c2fe-3828-4eea-91c3-1f6e0401aa7e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:41.908456 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001c2fe-3828-4eea-91c3-1f6e0401aa7e None admin] PATH_INFO: `/v3/groups/8763f0cf0bf74aa8a2f27c494160c014` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:41.910367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001c2fe-3828-4eea-91c3-1f6e0401aa7e None admin] RBAC: Authorizing `identity:get_group(group_id=8763f0cf0bf74aa8a2f27c494160c014)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:41.912484 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001c2fe-3828-4eea-91c3-1f6e0401aa7e None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:41.921485 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 190/368] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 26 05:29:41 2022] GET /identity/v3/groups/8763f0cf0bf74aa8a2f27c494160c014 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:41.923092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001c2fe-3828-4eea-91c3-1f6e0401aa7e None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:41.959917 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:42.032098 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7d32b-074e-4ffc-bdd4-9aecdb0f976e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:42.092339 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67635efa-b110-4613-8053-56c90b74a84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:42.093409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67635efa-b110-4613-8053-56c90b74a84c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:42.093901 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67635efa-b110-4613-8053-56c90b74a84c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:42.094380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67635efa-b110-4613-8053-56c90b74a84c None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:42.095586 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67635efa-b110-4613-8053-56c90b74a84c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:42.096115 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67635efa-b110-4613-8053-56c90b74a84c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:42.097021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001c2fe-3828-4eea-91c3-1f6e0401aa7e None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:42.099068 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67635efa-b110-4613-8053-56c90b74a84c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:42.120463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 179/369] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:42.121586 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67635efa-b110-4613-8053-56c90b74a84c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:42.162760 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c1aef-734d-4ade-90ed-3dcc6db47acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:42.163734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c1aef-734d-4ade-90ed-3dcc6db47acc None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:42.164176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c1aef-734d-4ade-90ed-3dcc6db47acc None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:42.164625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c1aef-734d-4ade-90ed-3dcc6db47acc None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:42.165767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c1aef-734d-4ade-90ed-3dcc6db47acc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:42.166268 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c1aef-734d-4ade-90ed-3dcc6db47acc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:42.169289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c1aef-734d-4ade-90ed-3dcc6db47acc None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:42.179507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:42.184016 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 191/370] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:42.185079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c1aef-734d-4ade-90ed-3dcc6db47acc None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:42.304031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67635efa-b110-4613-8053-56c90b74a84c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:42.339835 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:42.340343 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c1aef-734d-4ade-90ed-3dcc6db47acc None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:42.374371 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830f89e-6754-4f57-824d-c42475c1098e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:42.375745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830f89e-6754-4f57-824d-c42475c1098e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:42.376328 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830f89e-6754-4f57-824d-c42475c1098e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:42.376903 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830f89e-6754-4f57-824d-c42475c1098e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:42.378324 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830f89e-6754-4f57-824d-c42475c1098e None admin] RBAC: Adding query filter params (group.id=8763f0cf0bf74aa8a2f27c494160c014, role.id=bb454faf5dd643b892c6f9b670ad1f2d, scope.project.id=4e48d801558a426b8b994cb2bab00a20) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:42.378977 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830f89e-6754-4f57-824d-c42475c1098e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:42.380311 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0fe1f-22c0-41ac-a16c-500f5be251b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:42.380955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0fe1f-22c0-41ac-a16c-500f5be251b5 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:42.381253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0fe1f-22c0-41ac-a16c-500f5be251b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:42.381520 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0fe1f-22c0-41ac-a16c-500f5be251b5 None admin] PATH_INFO: `/v3/groups/8763f0cf0bf74aa8a2f27c494160c014` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:42.382293 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830f89e-6754-4f57-824d-c42475c1098e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:42.382670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a0fe1f-22c0-41ac-a16c-500f5be251b5 None admin] RBAC: Authorizing `identity:get_group(group_id=8763f0cf0bf74aa8a2f27c494160c014)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:42.384139 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a0fe1f-22c0-41ac-a16c-500f5be251b5 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:42.386238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 192/371] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 26 05:29:42 2022] GET /identity/v3/groups/8763f0cf0bf74aa8a2f27c494160c014 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:42.386915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0fe1f-22c0-41ac-a16c-500f5be251b5 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:42.413178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 180/372] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 26 05:29:42 2022] GET /identity/v3/role_assignments?group.id=8763f0cf0bf74aa8a2f27c494160c014&scope.project.id=4e48d801558a426b8b994cb2bab00a20&role.id=bb454faf5dd643b892c6f9b670ad1f2d => generated 260 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:42.413178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830f89e-6754-4f57-824d-c42475c1098e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:42.489623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:42.537173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0fe1f-22c0-41ac-a16c-500f5be251b5 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:42.579608 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd742f-4fe4-4526-87e3-f7b01cf610c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:42.580347 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd742f-4fe4-4526-87e3-f7b01cf610c7 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:42.581177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd742f-4fe4-4526-87e3-f7b01cf610c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:42.581177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd742f-4fe4-4526-87e3-f7b01cf610c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:42.581871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd742f-4fe4-4526-87e3-f7b01cf610c7 None admin] RBAC: Adding query filter params (group.id=8763f0cf0bf74aa8a2f27c494160c014, role.id=bb454faf5dd643b892c6f9b670ad1f2d, scope.project.id=b13122687f1640669b462d728fc006ce) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:42.582123 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd742f-4fe4-4526-87e3-f7b01cf610c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:42.583705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd742f-4fe4-4526-87e3-f7b01cf610c7 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:42.590961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 193/373] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 26 05:29:42 2022] GET /identity/v3/role_assignments?group.id=8763f0cf0bf74aa8a2f27c494160c014&scope.project.id=b13122687f1640669b462d728fc006ce&role.id=bb454faf5dd643b892c6f9b670ad1f2d => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:42.591888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd742f-4fe4-4526-87e3-f7b01cf610c7 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:42.632351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830f89e-6754-4f57-824d-c42475c1098e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:42.695292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9ef6a-d158-44bd-a016-8dbc1fda1dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:42.696922 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9ef6a-d158-44bd-a016-8dbc1fda1dcf None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:42.697762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9ef6a-d158-44bd-a016-8dbc1fda1dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:42.698487 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9ef6a-d158-44bd-a016-8dbc1fda1dcf None admin] PATH_INFO: `/v3/users/cf17484ab8464f7bb8cea72eee190a24` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:42.701122 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9ef6a-d158-44bd-a016-8dbc1fda1dcf None admin] RBAC: Authorizing `identity:get_user(user_id=cf17484ab8464f7bb8cea72eee190a24)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:42.704656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9ef6a-d158-44bd-a016-8dbc1fda1dcf None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:42.709248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 181/374] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:29:42 2022] GET /identity/v3/users/cf17484ab8464f7bb8cea72eee190a24 => generated 293 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:42.717415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9ef6a-d158-44bd-a016-8dbc1fda1dcf None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:42.717688 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd742f-4fe4-4526-87e3-f7b01cf610c7 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:42.751881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:42.758343 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e57720-505d-445e-9045-d51e60ac69a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:42.758979 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e57720-505d-445e-9045-d51e60ac69a0 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:42.759256 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e57720-505d-445e-9045-d51e60ac69a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:42.759516 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e57720-505d-445e-9045-d51e60ac69a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:42.834822 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:42.846622 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e57720-505d-445e-9045-d51e60ac69a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:42.848853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e57720-505d-445e-9045-d51e60ac69a0 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:42.944191 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9ef6a-d158-44bd-a016-8dbc1fda1dcf None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:42.974205 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 194/375] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:42.975372 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e57720-505d-445e-9045-d51e60ac69a0 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:43.014017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:43.033909 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f5ad9-24d7-4fd8-81b7-920cc0ce8ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:43.035150 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f5ad9-24d7-4fd8-81b7-920cc0ce8ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:43.035709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f5ad9-24d7-4fd8-81b7-920cc0ce8ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:43.036292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f5ad9-24d7-4fd8-81b7-920cc0ce8ae8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:43.037786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f5ad9-24d7-4fd8-81b7-920cc0ce8ae8 None admin] RBAC: Adding query filter params (group.id=8763f0cf0bf74aa8a2f27c494160c014, role.id=05e78a4a009c41488cf3d2c47e8cbe30, scope.project.id=b13122687f1640669b462d728fc006ce) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:43.038545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f5ad9-24d7-4fd8-81b7-920cc0ce8ae8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:43.041459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f5ad9-24d7-4fd8-81b7-920cc0ce8ae8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:43.052734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 182/376] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 26 05:29:42 2022] GET /identity/v3/role_assignments?group.id=8763f0cf0bf74aa8a2f27c494160c014&scope.project.id=b13122687f1640669b462d728fc006ce&role.id=05e78a4a009c41488cf3d2c47e8cbe30 => generated 260 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:43.054503 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f5ad9-24d7-4fd8-81b7-920cc0ce8ae8 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:43.100297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e57720-505d-445e-9045-d51e60ac69a0 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:43.141736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cc44d-9dfa-4ae6-b6e6-66d563737c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:43.142677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cc44d-9dfa-4ae6-b6e6-66d563737c67 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:43.143094 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cc44d-9dfa-4ae6-b6e6-66d563737c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:43.143511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cc44d-9dfa-4ae6-b6e6-66d563737c67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:43.144425 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1cc44d-9dfa-4ae6-b6e6-66d563737c67 None admin] RBAC: Adding query filter params (role.id=27a25b7936b8463c9e6d34d2cb49e61f, user.id=cf17484ab8464f7bb8cea72eee190a24, scope.system=all) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:43.144892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1cc44d-9dfa-4ae6-b6e6-66d563737c67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:43.146681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1cc44d-9dfa-4ae6-b6e6-66d563737c67 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:43.160658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 195/377] 10.222.0.57 () {56 vars in 1308 bytes} [Mon Sep 26 05:29:42 2022] GET /identity/v3/role_assignments?user.id=cf17484ab8464f7bb8cea72eee190a24&scope.system=all&role.id=27a25b7936b8463c9e6d34d2cb49e61f => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:43.293242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f5ad9-24d7-4fd8-81b7-920cc0ce8ae8 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:43.324320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:43.385370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c537d69-d47c-4ec4-8ccc-20bbe55a93d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:43.386452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c537d69-d47c-4ec4-8ccc-20bbe55a93d0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:43.386941 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c537d69-d47c-4ec4-8ccc-20bbe55a93d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:43.387378 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c537d69-d47c-4ec4-8ccc-20bbe55a93d0 None admin] PATH_INFO: `/v3/projects/fb1661fcfa594d16a5b5329ef20333a6` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:43.389452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c537d69-d47c-4ec4-8ccc-20bbe55a93d0 None admin] RBAC: Authorizing `identity:get_project(project_id=fb1661fcfa594d16a5b5329ef20333a6)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:43.392215 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c537d69-d47c-4ec4-8ccc-20bbe55a93d0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:43.399567 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 183/378] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:43 2022] GET /identity/v3/projects/fb1661fcfa594d16a5b5329ef20333a6 => generated 344 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:43.408583 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cc44d-9dfa-4ae6-b6e6-66d563737c67 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:43.546836 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:43.558479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cc44d-9dfa-4ae6-b6e6-66d563737c67 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:43.598669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a25db-f742-438c-9d3c-42a8e063e7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:43.599727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a25db-f742-438c-9d3c-42a8e063e7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:43.600197 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a25db-f742-438c-9d3c-42a8e063e7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:43.600657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a25db-f742-438c-9d3c-42a8e063e7b8 None admin] PATH_INFO: `/v3/groups/865cbf2df60b4542b47017e1e0c5737e` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:43.608442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a25db-f742-438c-9d3c-42a8e063e7b8 None admin] RBAC: Authorizing `identity:get_group(group_id=865cbf2df60b4542b47017e1e0c5737e)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:43.610417 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a25db-f742-438c-9d3c-42a8e063e7b8 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:43.612600 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 196/379] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 26 05:29:43 2022] GET /identity/v3/groups/865cbf2df60b4542b47017e1e0c5737e => generated 229 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:43.633494 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c537d69-d47c-4ec4-8ccc-20bbe55a93d0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:43.795866 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c537d69-d47c-4ec4-8ccc-20bbe55a93d0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:43.839707 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270e93c-fe5e-4190-b898-396c723c5c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:43.840457 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270e93c-fe5e-4190-b898-396c723c5c87 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:43.840738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270e93c-fe5e-4190-b898-396c723c5c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:43.840992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270e93c-fe5e-4190-b898-396c723c5c87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:43.841763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270e93c-fe5e-4190-b898-396c723c5c87 None admin] RBAC: Adding query filter params (group.id=865cbf2df60b4542b47017e1e0c5737e, role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=fb1661fcfa594d16a5b5329ef20333a6) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:43.842065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270e93c-fe5e-4190-b898-396c723c5c87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:43.843538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270e93c-fe5e-4190-b898-396c723c5c87 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:43.854352 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 184/380] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 26 05:29:43 2022] GET /identity/v3/role_assignments?group.id=865cbf2df60b4542b47017e1e0c5737e&scope.project.id=fb1661fcfa594d16a5b5329ef20333a6&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:45.727489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad6dff-5046-44da-b5e1-70b51c17ce0f None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:45.728549 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad6dff-5046-44da-b5e1-70b51c17ce0f None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:45.729159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad6dff-5046-44da-b5e1-70b51c17ce0f None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:45.731236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 197/381] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:29:45.737465 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f23345-150d-42d8-b5d7-e3fabe3e3b58 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:45.738422 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f23345-150d-42d8-b5d7-e3fabe3e3b58 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:45.738931 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f23345-150d-42d8-b5d7-e3fabe3e3b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:45.760377 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-20f23345-150d-42d8-b5d7-e3fabe3e3b58 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:45.762104 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-20f23345-150d-42d8-b5d7-e3fabe3e3b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:45.815664 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:45.849039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 185/382] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:29:46.128814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad6dff-5046-44da-b5e1-70b51c17ce0f None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:46.343839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad6dff-5046-44da-b5e1-70b51c17ce0f None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:46.432071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b6895b-2113-44bd-8b90-f01444d88d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:46.433055 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6895b-2113-44bd-8b90-f01444d88d18 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:46.433635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6895b-2113-44bd-8b90-f01444d88d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:46.434110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b6895b-2113-44bd-8b90-f01444d88d18 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:46.459016 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b6895b-2113-44bd-8b90-f01444d88d18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:46.465231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-31b6895b-2113-44bd-8b90-f01444d88d18 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:46.465231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:46.465231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:46.465231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:46.465231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:46.465231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:46.465231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:46.465231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:46.465231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:46.465231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:46.465231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:46.465231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:46.465231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:46.465231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:46.465231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:46.465231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:46.466199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:46.466199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:46.466199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:46.466199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:46.466199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:46.466199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:46.466199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:46.466199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:46.466199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:46.466199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:46.466199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:46.466199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:46.466199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:46.466199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:46.466199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:46.466199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:46.466983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:46.466983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:46.466983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:46.466983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:46.466983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:46.466983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:46.466983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:46.466983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:46.466983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:46.466983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:46.466983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:46.466983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:46.473337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 198/383] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:29:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:29:46.480573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f23345-150d-42d8-b5d7-e3fabe3e3b58 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:46.728716 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f23345-150d-42d8-b5d7-e3fabe3e3b58 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:46.775264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4924d-c84d-4b0a-86fc-c9d985dcf5bc None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:46.776382 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4924d-c84d-4b0a-86fc-c9d985dcf5bc None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:46.776850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4924d-c84d-4b0a-86fc-c9d985dcf5bc None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:46.778628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 199/384] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:29:46.799771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764788d9-9f24-487e-9795-86b72841e811 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:46.800191 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764788d9-9f24-487e-9795-86b72841e811 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:46.800557 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764788d9-9f24-487e-9795-86b72841e811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:46.827627 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c004e9-0fe4-4bae-b0c5-9e1993d12051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:46.829528 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c004e9-0fe4-4bae-b0c5-9e1993d12051 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:46.830182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c004e9-0fe4-4bae-b0c5-9e1993d12051 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:46.830846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c004e9-0fe4-4bae-b0c5-9e1993d12051 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:46.832251 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c004e9-0fe4-4bae-b0c5-9e1993d12051 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:46.832999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c004e9-0fe4-4bae-b0c5-9e1993d12051 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:46.835809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c004e9-0fe4-4bae-b0c5-9e1993d12051 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:46.848766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-764788d9-9f24-487e-9795-86b72841e811 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:46.857075 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-764788d9-9f24-487e-9795-86b72841e811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:46.862502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 186/385] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:29:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:46.872419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c004e9-0fe4-4bae-b0c5-9e1993d12051 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:46.887194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:46.964731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:46.981934 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 200/386] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:29:46.985206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8451e-4933-45bb-9924-75eb8b82809a None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:46.985206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8451e-4933-45bb-9924-75eb8b82809a None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:46.985665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8451e-4933-45bb-9924-75eb8b82809a None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:46.988703 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 201/387] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:46.997429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260afc0-b094-4584-9d98-1565dfa490bf None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:46.998225 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260afc0-b094-4584-9d98-1565dfa490bf None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:46.998906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260afc0-b094-4584-9d98-1565dfa490bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:47.067083 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-f260afc0-b094-4584-9d98-1565dfa490bf None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:47.069758 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f260afc0-b094-4584-9d98-1565dfa490bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:47.115355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c004e9-0fe4-4bae-b0c5-9e1993d12051 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:47.188902 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 202/388] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:29:47.203480 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bdbad-a478-4581-ad70-1944d95bffd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:47.204150 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bdbad-a478-4581-ad70-1944d95bffd3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:47.204428 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bdbad-a478-4581-ad70-1944d95bffd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:47.204703 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bdbad-a478-4581-ad70-1944d95bffd3 None admin] PATH_INFO: `/v3/groups/8763f0cf0bf74aa8a2f27c494160c014` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:47.205972 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bdbad-a478-4581-ad70-1944d95bffd3 None admin] RBAC: Authorizing `identity:get_group(group_id=8763f0cf0bf74aa8a2f27c494160c014)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:47.207457 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bdbad-a478-4581-ad70-1944d95bffd3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:47.209227 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 187/389] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 26 05:29:46 2022] GET /identity/v3/groups/8763f0cf0bf74aa8a2f27c494160c014 => generated 226 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:47.218157 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260afc0-b094-4584-9d98-1565dfa490bf None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:47.329809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:47.363858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260afc0-b094-4584-9d98-1565dfa490bf None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:47.414132 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075111b7-a49d-4b97-bcc5-46b71cbc687c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:47.414791 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075111b7-a49d-4b97-bcc5-46b71cbc687c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:47.415069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075111b7-a49d-4b97-bcc5-46b71cbc687c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:47.415344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075111b7-a49d-4b97-bcc5-46b71cbc687c None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:47.418169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075111b7-a49d-4b97-bcc5-46b71cbc687c None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:47.419825 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075111b7-a49d-4b97-bcc5-46b71cbc687c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:47.421630 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 203/390] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:47 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:47.428474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bdbad-a478-4581-ad70-1944d95bffd3 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:47.471416 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99e2d8-d91f-4042-97b9-05b5e8fbf238 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:47.472583 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99e2d8-d91f-4042-97b9-05b5e8fbf238 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:47.473333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99e2d8-d91f-4042-97b9-05b5e8fbf238 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:47.475943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 204/391] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:47.484663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38062e5-162d-4704-bcc8-4ecb2d7ecd93 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:47.485525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38062e5-162d-4704-bcc8-4ecb2d7ecd93 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:47.486203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38062e5-162d-4704-bcc8-4ecb2d7ecd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:47.514645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-f38062e5-162d-4704-bcc8-4ecb2d7ecd93 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:47.517299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f38062e5-162d-4704-bcc8-4ecb2d7ecd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:47.636093 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 205/392] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:29:47.696220 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38062e5-162d-4704-bcc8-4ecb2d7ecd93 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:47.727197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bdbad-a478-4581-ad70-1944d95bffd3 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:47.749755 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:47.824538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b47309-e5c6-4138-a247-cc7d40cca919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:47.826241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b47309-e5c6-4138-a247-cc7d40cca919 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:47.827050 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b47309-e5c6-4138-a247-cc7d40cca919 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:47.827740 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b47309-e5c6-4138-a247-cc7d40cca919 None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20/groups/8763f0cf0bf74aa8a2f27c494160c014/roles/05e78a4a009c41488cf3d2c47e8cbe30` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:47.833328 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b47309-e5c6-4138-a247-cc7d40cca919 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e48d801558a426b8b994cb2bab00a20, group_id=8763f0cf0bf74aa8a2f27c494160c014, role_id=05e78a4a009c41488cf3d2c47e8cbe30)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:47.836588 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b47309-e5c6-4138-a247-cc7d40cca919 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:47.861982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 188/393] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Sep 26 05:29:47 2022] PUT /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20/groups/8763f0cf0bf74aa8a2f27c494160c014/roles/05e78a4a009c41488cf3d2c47e8cbe30 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:29:47.862971 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b47309-e5c6-4138-a247-cc7d40cca919 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:47.876404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:47.923445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38062e5-162d-4704-bcc8-4ecb2d7ecd93 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:47.968794 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a54b3-f1b3-4424-bed1-0ca54d24fa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:47.969423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a54b3-f1b3-4424-bed1-0ca54d24fa7c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:47.969738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a54b3-f1b3-4424-bed1-0ca54d24fa7c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:47.970009 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a54b3-f1b3-4424-bed1-0ca54d24fa7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:47.985666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a54b3-f1b3-4424-bed1-0ca54d24fa7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:47.992771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-666a54b3-f1b3-4424-bed1-0ca54d24fa7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:47.992771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:47.992771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:47.992771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:47.992771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:47.992771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:47.992771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:47.992771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:47.992771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:47.992771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:47.992771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:47.992771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:47.992771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:47.992771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:47.992771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:47.992771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:47.993802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:47.993802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:47.993802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:47.993802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:47.993802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:47.993802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:47.993802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:47.993802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:47.993802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:47.993802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:47.993802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:47.993802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:47.993802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:47.993802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:47.993802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:47.993802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:47.994623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:47.994623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:47.994623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:47.994623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:47.994623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:47.994623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:47.994623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:47.994623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:47.994623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:47.994623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:47.994623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:47.994623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:47.997919 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 206/394] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:29:48.004781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a54b3-f1b3-4424-bed1-0ca54d24fa7c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:48.007880 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b47309-e5c6-4138-a247-cc7d40cca919 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:48.105791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ae518-43a4-4cda-af79-fb80f865ed07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:48.108164 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ae518-43a4-4cda-af79-fb80f865ed07 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:48.108701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ae518-43a4-4cda-af79-fb80f865ed07 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:48.109289 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ae518-43a4-4cda-af79-fb80f865ed07 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:48.139477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7ae518-43a4-4cda-af79-fb80f865ed07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:48.148336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-3c7ae518-43a4-4cda-af79-fb80f865ed07 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:48.148336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:48.148336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:48.148336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:48.148336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:48.148336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:48.148336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:48.148336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:48.148336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:48.148336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:48.148336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:48.148336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:48.148336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:48.148336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:48.148336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:48.148336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:48.149334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:48.149334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:48.149334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:48.149334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:48.149334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:48.149334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:48.149334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:48.149334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:48.149334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:48.149334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:48.149334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:48.149334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:48.149334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:48.149334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:48.149334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:48.149334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:48.150136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:48.150136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:48.150136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:48.150136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:48.150136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:48.150136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:48.150136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:48.150136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:48.150136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:48.150136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:48.150136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:48.150136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:48.150136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 189/395] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:29:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:48.157473 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ae518-43a4-4cda-af79-fb80f865ed07 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:48.229179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:48.253676 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a54b3-f1b3-4424-bed1-0ca54d24fa7c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:48.306872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:48.316252 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745f207-4698-476a-960a-20542a41192c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:48.317316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745f207-4698-476a-960a-20542a41192c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:48.318389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745f207-4698-476a-960a-20542a41192c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:48.318389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745f207-4698-476a-960a-20542a41192c None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:48.319455 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745f207-4698-476a-960a-20542a41192c None admin] RBAC: Adding query filter params (name=member) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:48.319991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745f207-4698-476a-960a-20542a41192c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:48.322394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745f207-4698-476a-960a-20542a41192c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:48.342956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 207/396] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:48.342956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745f207-4698-476a-960a-20542a41192c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:48.375818 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ae518-43a4-4cda-af79-fb80f865ed07 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:48.427309 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1ea34-1461-452d-ac37-f79f877143db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:48.428703 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1ea34-1461-452d-ac37-f79f877143db None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:48.429017 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1ea34-1461-452d-ac37-f79f877143db None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:48.429287 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1ea34-1461-452d-ac37-f79f877143db None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:48.430265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1ea34-1461-452d-ac37-f79f877143db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:48.430563 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1ea34-1461-452d-ac37-f79f877143db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:48.432140 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1ea34-1461-452d-ac37-f79f877143db None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:48.443764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 190/397] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:29:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:48.446308 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f3f92-6fb7-42cb-bbf0-aa85ca8d7a20 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:48.447348 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f3f92-6fb7-42cb-bbf0-aa85ca8d7a20 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:48.447684 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f3f92-6fb7-42cb-bbf0-aa85ca8d7a20 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:48.450452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 191/398] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:48.451671 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f3f92-6fb7-42cb-bbf0-aa85ca8d7a20 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:48.520714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745f207-4698-476a-960a-20542a41192c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:48.559658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc1a0e-aa41-459f-b78e-bf5cbeced364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:48.560305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc1a0e-aa41-459f-b78e-bf5cbeced364 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:48.560508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:48.560609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc1a0e-aa41-459f-b78e-bf5cbeced364 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:48.560841 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc1a0e-aa41-459f-b78e-bf5cbeced364 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:48.575110 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f3f92-6fb7-42cb-bbf0-aa85ca8d7a20 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:48.580062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc1a0e-aa41-459f-b78e-bf5cbeced364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:48.590777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-76fc1a0e-aa41-459f-b78e-bf5cbeced364 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:48.590777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:48.590777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:48.590777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:48.590777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:48.590777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:48.590777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:48.590777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:48.590777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:48.590777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:48.590777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:48.590777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:48.590777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:48.590777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:48.590777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:48.590777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:48.591733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:48.591733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:48.591733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:48.591733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:48.591733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:48.591733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:48.591733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:48.591733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:48.591733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:48.591733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:48.591733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:48.591733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:48.591733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:48.591733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:48.591733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:48.591733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:48.592510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:48.592510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:48.592510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:48.592510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:48.592510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:48.592510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:48.592510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:48.592510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:48.592510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:48.592510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:48.592510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:48.592510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:48.593114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 208/399] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:48.594205 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc1a0e-aa41-459f-b78e-bf5cbeced364 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:48.618588 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43533e0d-6d39-4bcb-af74-1302ac93dd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:48.619248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43533e0d-6d39-4bcb-af74-1302ac93dd3a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:48.619533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43533e0d-6d39-4bcb-af74-1302ac93dd3a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:48.619810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43533e0d-6d39-4bcb-af74-1302ac93dd3a None admin] PATH_INFO: `/v3/groups/8763f0cf0bf74aa8a2f27c494160c014` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:48.621104 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43533e0d-6d39-4bcb-af74-1302ac93dd3a None admin] RBAC: Authorizing `identity:get_group(group_id=8763f0cf0bf74aa8a2f27c494160c014)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:48.622863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43533e0d-6d39-4bcb-af74-1302ac93dd3a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:48.624260 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 192/400] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 26 05:29:48 2022] GET /identity/v3/groups/8763f0cf0bf74aa8a2f27c494160c014 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:48.626551 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13d1dd-e76f-47af-95fc-4a3bc42b3efd None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:48.627357 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13d1dd-e76f-47af-95fc-4a3bc42b3efd None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:48.627821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13d1dd-e76f-47af-95fc-4a3bc42b3efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:48.647989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-9a13d1dd-e76f-47af-95fc-4a3bc42b3efd None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:48.649420 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9a13d1dd-e76f-47af-95fc-4a3bc42b3efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:48.690209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:48.730059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 193/401] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:48.731478 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a13d1dd-e76f-47af-95fc-4a3bc42b3efd None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:48.784202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc1a0e-aa41-459f-b78e-bf5cbeced364 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:48.856927 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a331071-9365-48d8-9dc5-d8a092d17eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:48.859195 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a331071-9365-48d8-9dc5-d8a092d17eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:48.859855 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a331071-9365-48d8-9dc5-d8a092d17eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:48.860524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a331071-9365-48d8-9dc5-d8a092d17eb1 None admin] PATH_INFO: `/v3/groups/8763f0cf0bf74aa8a2f27c494160c014` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:48.862638 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a331071-9365-48d8-9dc5-d8a092d17eb1 None admin] RBAC: Authorizing `identity:get_group(group_id=8763f0cf0bf74aa8a2f27c494160c014)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:48.865366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a331071-9365-48d8-9dc5-d8a092d17eb1 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:48.869951 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 209/402] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 26 05:29:48 2022] GET /identity/v3/groups/8763f0cf0bf74aa8a2f27c494160c014 => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:48.871746 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a331071-9365-48d8-9dc5-d8a092d17eb1 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:48.913803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:48.921411 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a13d1dd-e76f-47af-95fc-4a3bc42b3efd None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:48.993487 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b13d9-c61c-41b6-aa5b-09cd94abeb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:48.995005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b13d9-c61c-41b6-aa5b-09cd94abeb0d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:48.995573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b13d9-c61c-41b6-aa5b-09cd94abeb0d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:48.996184 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b13d9-c61c-41b6-aa5b-09cd94abeb0d None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:48.997743 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b13d9-c61c-41b6-aa5b-09cd94abeb0d None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:48.998358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b13d9-c61c-41b6-aa5b-09cd94abeb0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:49.000786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b13d9-c61c-41b6-aa5b-09cd94abeb0d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:49.021356 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 194/403] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:49.023225 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b13d9-c61c-41b6-aa5b-09cd94abeb0d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:49.089182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a331071-9365-48d8-9dc5-d8a092d17eb1 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:49.139115 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:49.157438 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f873a8-5d14-4e64-93ae-9bcc323be232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:49.158647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f873a8-5d14-4e64-93ae-9bcc323be232 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:49.159217 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f873a8-5d14-4e64-93ae-9bcc323be232 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:49.160173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f873a8-5d14-4e64-93ae-9bcc323be232 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:49.162125 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f873a8-5d14-4e64-93ae-9bcc323be232 None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:49.164803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f873a8-5d14-4e64-93ae-9bcc323be232 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:49.173718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 210/404] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:48 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:49.175384 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f873a8-5d14-4e64-93ae-9bcc323be232 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:49.234558 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b13d9-c61c-41b6-aa5b-09cd94abeb0d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:49.291639 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ca27d-c281-4c29-ab51-f5536549742b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:49.292522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ca27d-c281-4c29-ab51-f5536549742b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:49.293025 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ca27d-c281-4c29-ab51-f5536549742b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:49.293497 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ca27d-c281-4c29-ab51-f5536549742b None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:49.295981 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ca27d-c281-4c29-ab51-f5536549742b None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:49.298882 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ca27d-c281-4c29-ab51-f5536549742b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:49.302095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 195/405] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:49 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:49.303235 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ca27d-c281-4c29-ab51-f5536549742b None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:49.337259 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:49.351328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f873a8-5d14-4e64-93ae-9bcc323be232 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:49.403081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49a1d4-a4a1-45b1-8806-2b6427294f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:49.404458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49a1d4-a4a1-45b1-8806-2b6427294f8e None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:49.404933 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49a1d4-a4a1-45b1-8806-2b6427294f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:49.405497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49a1d4-a4a1-45b1-8806-2b6427294f8e None admin] PATH_INFO: `/v3/groups/8763f0cf0bf74aa8a2f27c494160c014` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:49.407431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49a1d4-a4a1-45b1-8806-2b6427294f8e None admin] RBAC: Authorizing `identity:get_group(group_id=8763f0cf0bf74aa8a2f27c494160c014)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:49.410146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49a1d4-a4a1-45b1-8806-2b6427294f8e None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:49.418031 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 211/406] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 26 05:29:49 2022] GET /identity/v3/groups/8763f0cf0bf74aa8a2f27c494160c014 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:49.418031 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49a1d4-a4a1-45b1-8806-2b6427294f8e None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:49.472511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ca27d-c281-4c29-ab51-f5536549742b None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:49.507344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:49.530136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ac3b0-d9b0-4310-804d-ec02f0a201c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:49.531141 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ac3b0-d9b0-4310-804d-ec02f0a201c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:49.531603 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ac3b0-d9b0-4310-804d-ec02f0a201c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:49.532083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ac3b0-d9b0-4310-804d-ec02f0a201c3 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce/groups/8763f0cf0bf74aa8a2f27c494160c014/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:49.534408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ac3b0-d9b0-4310-804d-ec02f0a201c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b13122687f1640669b462d728fc006ce, group_id=8763f0cf0bf74aa8a2f27c494160c014, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:49.536209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ac3b0-d9b0-4310-804d-ec02f0a201c3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:49.554163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 196/407] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Sep 26 05:29:49 2022] PUT /identity/v3/projects/b13122687f1640669b462d728fc006ce/groups/8763f0cf0bf74aa8a2f27c494160c014/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:49.556110 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ac3b0-d9b0-4310-804d-ec02f0a201c3 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:49.594625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49a1d4-a4a1-45b1-8806-2b6427294f8e None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:49.669239 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bf6d4-3f0c-4cf9-97b7-1ffec40a376e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:49.669760 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bf6d4-3f0c-4cf9-97b7-1ffec40a376e None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:49.670040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bf6d4-3f0c-4cf9-97b7-1ffec40a376e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:49.670310 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bf6d4-3f0c-4cf9-97b7-1ffec40a376e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:49.682988 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19bf6d4-3f0c-4cf9-97b7-1ffec40a376e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:49.689853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-f19bf6d4-3f0c-4cf9-97b7-1ffec40a376e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:29:49.689853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:49.689853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:49.689853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:49.689853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:49.689853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:49.689853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:49.689853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:49.689853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:49.689853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:49.689853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:49.689853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:49.689853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:49.689853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:49.689853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:49.689853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:49.697741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:49.697741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:49.697741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:49.697741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:49.697741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:49.697741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:49.697741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:49.697741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:49.697741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:49.697741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:49.697741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:49.697741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:49.697741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:49.697741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:49.697741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:49.697741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:49.698597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:49.698597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:49.698597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:49.698597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:49.698597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:49.698597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:49.698597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:49.698597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:49.698597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:49.698597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:49.698597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:29:49.698597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:49.698597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 212/408] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:29:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:49.698597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bf6d4-3f0c-4cf9-97b7-1ffec40a376e None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:49.716679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:49.771184 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ac3b0-d9b0-4310-804d-ec02f0a201c3 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:49.816748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8bff2-ed43-43dd-aba7-8b89c0e0d608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:49.819461 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8bff2-ed43-43dd-aba7-8b89c0e0d608 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:49.820111 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8bff2-ed43-43dd-aba7-8b89c0e0d608 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:49.820705 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8bff2-ed43-43dd-aba7-8b89c0e0d608 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce/groups/8763f0cf0bf74aa8a2f27c494160c014/roles/05e78a4a009c41488cf3d2c47e8cbe30` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:49.827348 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8bff2-ed43-43dd-aba7-8b89c0e0d608 None admin] RBAC: Authorizing `identity:create_grant(project_id=b13122687f1640669b462d728fc006ce, group_id=8763f0cf0bf74aa8a2f27c494160c014, role_id=05e78a4a009c41488cf3d2c47e8cbe30)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:49.830591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8bff2-ed43-43dd-aba7-8b89c0e0d608 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:49.863382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 197/409] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Sep 26 05:29:49 2022] PUT /identity/v3/projects/b13122687f1640669b462d728fc006ce/groups/8763f0cf0bf74aa8a2f27c494160c014/roles/05e78a4a009c41488cf3d2c47e8cbe30 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:49.865933 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8bff2-ed43-43dd-aba7-8b89c0e0d608 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:49.885384 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bf6d4-3f0c-4cf9-97b7-1ffec40a376e None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:49.909380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:49.934035 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b1310-b0d4-4c3a-a7d6-75412aead534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:49.935126 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b1310-b0d4-4c3a-a7d6-75412aead534 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:49.935631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b1310-b0d4-4c3a-a7d6-75412aead534 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:49.936109 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b1310-b0d4-4c3a-a7d6-75412aead534 None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:49.938118 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b1310-b0d4-4c3a-a7d6-75412aead534 None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:49.940646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b1310-b0d4-4c3a-a7d6-75412aead534 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:49.942905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 213/410] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:49 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:49.951799 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b1310-b0d4-4c3a-a7d6-75412aead534 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:50.144822 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8bff2-ed43-43dd-aba7-8b89c0e0d608 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:50.221256 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d6937-04fa-4e0e-a3d7-395fff302ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:50.222625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d6937-04fa-4e0e-a3d7-395fff302ccb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:50.223918 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d6937-04fa-4e0e-a3d7-395fff302ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:50.224196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d6937-04fa-4e0e-a3d7-395fff302ccb None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:50.224914 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d6937-04fa-4e0e-a3d7-395fff302ccb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:50.226953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d6937-04fa-4e0e-a3d7-395fff302ccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:50.227772 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d6937-04fa-4e0e-a3d7-395fff302ccb None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:50.238456 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b1310-b0d4-4c3a-a7d6-75412aead534 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:50.240192 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:50.254698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 198/411] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:50.264315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d6937-04fa-4e0e-a3d7-395fff302ccb None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:50.291728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35660f-2866-4a7b-824e-550229ceb639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:50.293386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35660f-2866-4a7b-824e-550229ceb639 None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:50.294442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35660f-2866-4a7b-824e-550229ceb639 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:50.295309 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35660f-2866-4a7b-824e-550229ceb639 None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20/groups/8763f0cf0bf74aa8a2f27c494160c014/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:50.299433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35660f-2866-4a7b-824e-550229ceb639 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e48d801558a426b8b994cb2bab00a20, group_id=8763f0cf0bf74aa8a2f27c494160c014, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:50.302513 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35660f-2866-4a7b-824e-550229ceb639 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:50.340640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 214/412] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Sep 26 05:29:49 2022] PUT /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20/groups/8763f0cf0bf74aa8a2f27c494160c014/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:50.398900 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d6937-04fa-4e0e-a3d7-395fff302ccb None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:50.440234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a79e5-146c-4a3c-b4a8-029359bbb40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:50.440913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a79e5-146c-4a3c-b4a8-029359bbb40c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:50.441222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a79e5-146c-4a3c-b4a8-029359bbb40c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:50.441496 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a79e5-146c-4a3c-b4a8-029359bbb40c None admin] PATH_INFO: `/v3/groups/865cbf2df60b4542b47017e1e0c5737e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:50.442871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a79e5-146c-4a3c-b4a8-029359bbb40c None admin] RBAC: Authorizing `identity:get_group(group_id=865cbf2df60b4542b47017e1e0c5737e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:50.444400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a79e5-146c-4a3c-b4a8-029359bbb40c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:50.446518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 199/413] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 26 05:29:50 2022] GET /identity/v3/groups/865cbf2df60b4542b47017e1e0c5737e => generated 229 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:50.454946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35660f-2866-4a7b-824e-550229ceb639 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:50.607605 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:50.723792 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35660f-2866-4a7b-824e-550229ceb639 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:50.778738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2884b87-33d8-428e-a5ad-5e640ca2bc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:50.779389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2884b87-33d8-428e-a5ad-5e640ca2bc0e None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:50.779672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2884b87-33d8-428e-a5ad-5e640ca2bc0e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:50.779948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2884b87-33d8-428e-a5ad-5e640ca2bc0e None admin] PATH_INFO: `/v3/projects/fb1661fcfa594d16a5b5329ef20333a6` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:50.781157 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2884b87-33d8-428e-a5ad-5e640ca2bc0e None admin] RBAC: Authorizing `identity:get_project(project_id=fb1661fcfa594d16a5b5329ef20333a6)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:50.782644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2884b87-33d8-428e-a5ad-5e640ca2bc0e None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:50.784670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 215/414] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:50 2022] GET /identity/v3/projects/fb1661fcfa594d16a5b5329ef20333a6 => generated 344 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:50.793268 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a79e5-146c-4a3c-b4a8-029359bbb40c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:50.907285 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:50.947308 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a79e5-146c-4a3c-b4a8-029359bbb40c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:50.988798 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a60965-621f-43e7-8056-6459648f5294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:50.989526 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a60965-621f-43e7-8056-6459648f5294 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:50.989804 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a60965-621f-43e7-8056-6459648f5294 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:50.990077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a60965-621f-43e7-8056-6459648f5294 None admin] PATH_INFO: `/v3/projects/fb1661fcfa594d16a5b5329ef20333a6/groups/865cbf2df60b4542b47017e1e0c5737e/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:50.992540 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a60965-621f-43e7-8056-6459648f5294 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb1661fcfa594d16a5b5329ef20333a6, group_id=865cbf2df60b4542b47017e1e0c5737e, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:50.994050 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a60965-621f-43e7-8056-6459648f5294 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:51.022605 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 200/415] 10.222.0.57 () {58 vars in 1442 bytes} [Mon Sep 26 05:29:50 2022] PUT /identity/v3/projects/fb1661fcfa594d16a5b5329ef20333a6/groups/865cbf2df60b4542b47017e1e0c5737e/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:29:51.618328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85622154-54d7-499f-85a9-6ff3344d60db None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:51.618867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85622154-54d7-499f-85a9-6ff3344d60db None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:51.618867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85622154-54d7-499f-85a9-6ff3344d60db None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:51.620787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 216/416] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:51.626941 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bc479-b5c2-4d8a-8f15-691336c85173 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:51.627457 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bc479-b5c2-4d8a-8f15-691336c85173 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:51.627457 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bc479-b5c2-4d8a-8f15-691336c85173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:51.653019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-266bc479-b5c2-4d8a-8f15-691336c85173 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:51.654326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-266bc479-b5c2-4d8a-8f15-691336c85173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:51.755542 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 201/417] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:52.188385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85622154-54d7-499f-85a9-6ff3344d60db None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:52.357822 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85622154-54d7-499f-85a9-6ff3344d60db None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:52.381489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:52.412624 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695caea-1a7f-4a37-9db2-19ddba05b507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:52.417547 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695caea-1a7f-4a37-9db2-19ddba05b507 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:52.417966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695caea-1a7f-4a37-9db2-19ddba05b507 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:52.418248 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695caea-1a7f-4a37-9db2-19ddba05b507 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:52.432875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695caea-1a7f-4a37-9db2-19ddba05b507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:52.442360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-4695caea-1a7f-4a37-9db2-19ddba05b507 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:52.442360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:52.442360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:52.442360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:52.442360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:52.442360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:52.442360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:52.442360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:52.442360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:52.442360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:52.442360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:52.442360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:52.442360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:52.442360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:52.442360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:52.442360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:52.443514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:52.443514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:52.443514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:52.443514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:52.443514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:52.443514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:52.443514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:52.443514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:52.443514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:52.443514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:52.443514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:52.443514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:52.443514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:52.443514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:52.443514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:52.443514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:52.444899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:52.444899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:52.444899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:52.444899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:52.444899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:52.444899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:52.444899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:52.444899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:52.444899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:52.444899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:52.444899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:52.444899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:29:52.444899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 217/418] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:29:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:52.449807 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bc479-b5c2-4d8a-8f15-691336c85173 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:52.560437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:52.633827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bc479-b5c2-4d8a-8f15-691336c85173 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:52.687999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4155d-ee53-4cf1-b424-abb27d912914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:52.688644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4155d-ee53-4cf1-b424-abb27d912914 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:52.688918 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4155d-ee53-4cf1-b424-abb27d912914 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:52.689208 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4155d-ee53-4cf1-b424-abb27d912914 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:52.689888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4155d-ee53-4cf1-b424-abb27d912914 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:52.690185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4155d-ee53-4cf1-b424-abb27d912914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:52.691859 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4155d-ee53-4cf1-b424-abb27d912914 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:52.706103 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 202/419] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:29:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:52.713328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695caea-1a7f-4a37-9db2-19ddba05b507 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:52.896778 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051c56f-bb98-43d8-9512-6b84f0814b29 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:52.897336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051c56f-bb98-43d8-9512-6b84f0814b29 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:52.897757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051c56f-bb98-43d8-9512-6b84f0814b29 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:52.899593 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 203/420] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:52.906657 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752905a-97ba-4397-ad46-490879d3aa96 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:52.907173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752905a-97ba-4397-ad46-490879d3aa96 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:52.907590 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752905a-97ba-4397-ad46-490879d3aa96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:52.964146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7752905a-97ba-4397-ad46-490879d3aa96 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:52.966146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7752905a-97ba-4397-ad46-490879d3aa96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:52.976591 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695caea-1a7f-4a37-9db2-19ddba05b507 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:53.070003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d37de-61b2-4e98-959b-cf6d12c40061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:53.070898 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d37de-61b2-4e98-959b-cf6d12c40061 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:53.071349 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d37de-61b2-4e98-959b-cf6d12c40061 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:53.071767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d37de-61b2-4e98-959b-cf6d12c40061 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:53.130195 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:53.145317 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 204/421] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:53.147249 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27724585-8763-4be5-a287-8bc8fcae5b86 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:53.147673 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27724585-8763-4be5-a287-8bc8fcae5b86 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:53.148075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27724585-8763-4be5-a287-8bc8fcae5b86 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:53.150522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 205/422] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:53.156111 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d816b-c762-4db7-92fb-d42c3c3b11f6 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:53.156384 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d816b-c762-4db7-92fb-d42c3c3b11f6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:53.156836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d816b-c762-4db7-92fb-d42c3c3b11f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:53.176588 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8d7d816b-c762-4db7-92fb-d42c3c3b11f6 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:53.178092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8d7d816b-c762-4db7-92fb-d42c3c3b11f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:53.184089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d37de-61b2-4e98-959b-cf6d12c40061 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:53.187192 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d37de-61b2-4e98-959b-cf6d12c40061 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:53.250750 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:53.307537 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 206/423] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:29:53.334836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33998092-6c0d-4fc3-9630-7e2820d1c1f9 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:53.335208 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33998092-6c0d-4fc3-9630-7e2820d1c1f9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:53.335481 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33998092-6c0d-4fc3-9630-7e2820d1c1f9 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:53.336872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 207/424] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:29:53.342974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a00a50-1b4a-491e-9d54-19e8ec1d00c4 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:53.343263 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a00a50-1b4a-491e-9d54-19e8ec1d00c4 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:53.343517 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a00a50-1b4a-491e-9d54-19e8ec1d00c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:53.363544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c0a00a50-1b4a-491e-9d54-19e8ec1d00c4 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:53.364843 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c0a00a50-1b4a-491e-9d54-19e8ec1d00c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:53.382566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 218/425] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 670 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:53.392477 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d37de-61b2-4e98-959b-cf6d12c40061 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:53.449952 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 208/426] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:53.541221 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a00a50-1b4a-491e-9d54-19e8ec1d00c4 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:53.558873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:53.623226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d37de-61b2-4e98-959b-cf6d12c40061 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:53.687483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe74c6a-772a-4084-a2b0-4c1b39852a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:53.688505 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe74c6a-772a-4084-a2b0-4c1b39852a4f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:53.689004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe74c6a-772a-4084-a2b0-4c1b39852a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:53.689495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe74c6a-772a-4084-a2b0-4c1b39852a4f None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:53.691017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe74c6a-772a-4084-a2b0-4c1b39852a4f None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:53.693423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe74c6a-772a-4084-a2b0-4c1b39852a4f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:53.695869 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 219/427] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:53 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:53.704387 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe74c6a-772a-4084-a2b0-4c1b39852a4f None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:53.753955 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a00a50-1b4a-491e-9d54-19e8ec1d00c4 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:53.777569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:53.818695 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e0531-0dff-443b-b119-49376169dbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:53.819355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e0531-0dff-443b-b119-49376169dbbc None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:53.819638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e0531-0dff-443b-b119-49376169dbbc None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:53.820056 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e0531-0dff-443b-b119-49376169dbbc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:53.834893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e0531-0dff-443b-b119-49376169dbbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:53.841315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-055e0531-0dff-443b-b119-49376169dbbc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:53.841315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:53.841315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:53.841315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:53.841315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:53.841315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:53.841315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:53.841315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:53.841315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:53.841315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:53.841315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:53.841315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:53.841315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:53.841315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:53.841315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:53.841315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:53.842255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:53.842255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:53.842255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:53.842255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:53.842255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:53.842255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:53.842255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:53.842255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:53.842255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:53.842255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:53.842255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:53.842255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:53.842255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:53.842255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:53.842255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:53.842255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:53.843031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:53.843031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:53.843031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:53.843031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:53.843031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:53.843031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:53.843031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:53.843031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:53.843031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:53.843031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:53.843031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:53.843031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:53.845888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 209/428] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:29:53.847399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e0531-0dff-443b-b119-49376169dbbc None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:53.867550 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe74c6a-772a-4084-a2b0-4c1b39852a4f None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:53.919508 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbeaeb-5f6e-4ef1-b602-e53fdcb9178c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:53.920552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbeaeb-5f6e-4ef1-b602-e53fdcb9178c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:53.921033 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbeaeb-5f6e-4ef1-b602-e53fdcb9178c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:53.921544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbeaeb-5f6e-4ef1-b602-e53fdcb9178c None admin] PATH_INFO: `/v3/groups/8763f0cf0bf74aa8a2f27c494160c014` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:53.923465 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbeaeb-5f6e-4ef1-b602-e53fdcb9178c None admin] RBAC: Authorizing `identity:get_group(group_id=8763f0cf0bf74aa8a2f27c494160c014)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:53.926177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbeaeb-5f6e-4ef1-b602-e53fdcb9178c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:53.933179 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 220/429] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 26 05:29:53 2022] GET /identity/v3/groups/8763f0cf0bf74aa8a2f27c494160c014 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:53.933179 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbeaeb-5f6e-4ef1-b602-e53fdcb9178c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:53.954258 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:53.983931 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e0531-0dff-443b-b119-49376169dbbc None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:54.034712 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d764d-6729-4e85-b607-0b007b96e60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:54.035679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d764d-6729-4e85-b607-0b007b96e60f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:54.036167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d764d-6729-4e85-b607-0b007b96e60f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:54.036605 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d764d-6729-4e85-b607-0b007b96e60f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:54.050573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d764d-6729-4e85-b607-0b007b96e60f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:54.055974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-ce1d764d-6729-4e85-b607-0b007b96e60f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:54.055974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:54.055974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:54.055974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:54.055974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:54.055974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:54.055974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:54.055974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:54.055974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:54.055974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:54.055974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:54.055974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:54.055974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:54.055974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:54.055974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:54.055974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:54.057888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:54.057888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:54.057888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:54.057888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:54.057888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:54.057888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:54.057888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:54.057888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:54.057888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:54.057888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:54.057888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:54.057888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:54.057888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:54.057888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:54.057888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:54.057888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:54.059632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:54.059632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:54.059632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:54.059632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:54.059632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:54.059632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:54.059632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:54.059632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:54.059632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:54.059632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:54.059632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 05:29:54.059632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:54.063282 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 210/430] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:54.064272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d764d-6729-4e85-b607-0b007b96e60f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:54.089429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbeaeb-5f6e-4ef1-b602-e53fdcb9178c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:54.095375 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:54.130292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e3d9b-c59f-43e8-8a83-3da2e1e4c1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:54.130952 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e3d9b-c59f-43e8-8a83-3da2e1e4c1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:54.131212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e3d9b-c59f-43e8-8a83-3da2e1e4c1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:54.131487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e3d9b-c59f-43e8-8a83-3da2e1e4c1f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:54.132201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e3d9b-c59f-43e8-8a83-3da2e1e4c1f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:54.132508 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e3d9b-c59f-43e8-8a83-3da2e1e4c1f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:54.133990 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e3d9b-c59f-43e8-8a83-3da2e1e4c1f6 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:54.144479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 221/431] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:54.145799 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e3d9b-c59f-43e8-8a83-3da2e1e4c1f6 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:54.183963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d764d-6729-4e85-b607-0b007b96e60f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:54.222670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d085e6-358f-4aeb-8b3f-c711ee86cf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:54.223328 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d085e6-358f-4aeb-8b3f-c711ee86cf35 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:54.223599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d085e6-358f-4aeb-8b3f-c711ee86cf35 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:54.223870 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d085e6-358f-4aeb-8b3f-c711ee86cf35 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:54.235955 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d085e6-358f-4aeb-8b3f-c711ee86cf35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:54.239203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:54.240911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-10d085e6-358f-4aeb-8b3f-c711ee86cf35 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:54.240911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:54.240911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:54.240911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:54.240911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:54.240911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:54.240911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:54.240911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:54.240911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:54.240911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:54.240911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:54.240911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:54.240911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:54.240911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:54.240911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:54.240911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:54.242091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:54.242091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:54.242091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:54.242091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:54.242091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:54.242091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:54.242091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:54.242091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:54.242091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:54.242091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:54.242091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:54.242091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:54.242091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:54.242091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:54.242091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:54.242091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:54.243686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:54.243686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:54.243686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:54.243686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:54.243686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:54.243686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:54.243686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:54.243686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:54.243686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:54.243686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:54.243686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:29:54.243686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:54.243686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 211/432] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:29:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:54.243686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d085e6-358f-4aeb-8b3f-c711ee86cf35 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:54.277942 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e3d9b-c59f-43e8-8a83-3da2e1e4c1f6 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:54.315697 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d42384-7aac-4b7d-9458-2f2e1f7e7db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:54.316318 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d42384-7aac-4b7d-9458-2f2e1f7e7db7 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:54.316597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d42384-7aac-4b7d-9458-2f2e1f7e7db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:54.316861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d42384-7aac-4b7d-9458-2f2e1f7e7db7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:54.317627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d42384-7aac-4b7d-9458-2f2e1f7e7db7 None admin] RBAC: Adding query filter params (group.id=8763f0cf0bf74aa8a2f27c494160c014, role.id=05e78a4a009c41488cf3d2c47e8cbe30, scope.project.id=4e48d801558a426b8b994cb2bab00a20) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:54.317930 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d42384-7aac-4b7d-9458-2f2e1f7e7db7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:54.319387 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d42384-7aac-4b7d-9458-2f2e1f7e7db7 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:54.325808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 222/433] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 26 05:29:54 2022] GET /identity/v3/role_assignments?group.id=8763f0cf0bf74aa8a2f27c494160c014&scope.project.id=4e48d801558a426b8b994cb2bab00a20&role.id=05e78a4a009c41488cf3d2c47e8cbe30 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:54.327163 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ca940-9242-4a23-9fe9-6a06dc87023d None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:54.327444 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ca940-9242-4a23-9fe9-6a06dc87023d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:54.327710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ca940-9242-4a23-9fe9-6a06dc87023d None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:54.328982 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 223/434] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:29:54.329786 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ca940-9242-4a23-9fe9-6a06dc87023d None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:54.364780 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:54.381354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d085e6-358f-4aeb-8b3f-c711ee86cf35 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:54.422559 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af326a23-b2f7-4d3e-b63a-0f07035515f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:54.423187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af326a23-b2f7-4d3e-b63a-0f07035515f2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:54.423461 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af326a23-b2f7-4d3e-b63a-0f07035515f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:54.423727 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af326a23-b2f7-4d3e-b63a-0f07035515f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:54.424413 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af326a23-b2f7-4d3e-b63a-0f07035515f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:54.424708 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af326a23-b2f7-4d3e-b63a-0f07035515f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:54.426239 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af326a23-b2f7-4d3e-b63a-0f07035515f2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:54.443986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 212/435] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:29:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:54.443986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af326a23-b2f7-4d3e-b63a-0f07035515f2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:54.463935 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ca940-9242-4a23-9fe9-6a06dc87023d None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:54.506467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0792e-0b4d-4cc7-af98-93ae4558da37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:54.507096 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0792e-0b4d-4cc7-af98-93ae4558da37 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:54.507372 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0792e-0b4d-4cc7-af98-93ae4558da37 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:54.507742 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0792e-0b4d-4cc7-af98-93ae4558da37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:54.512518 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:54.591447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af326a23-b2f7-4d3e-b63a-0f07035515f2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:54.600219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0792e-0b4d-4cc7-af98-93ae4558da37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:54.601775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0792e-0b4d-4cc7-af98-93ae4558da37 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:54.635096 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85bb21-634b-42bf-81b8-64e1439e2195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:54.635625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85bb21-634b-42bf-81b8-64e1439e2195 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:54.635913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85bb21-634b-42bf-81b8-64e1439e2195 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:54.636210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85bb21-634b-42bf-81b8-64e1439e2195 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:54.636878 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85bb21-634b-42bf-81b8-64e1439e2195 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:54.637199 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85bb21-634b-42bf-81b8-64e1439e2195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:54.638751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85bb21-634b-42bf-81b8-64e1439e2195 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:54.652509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 213/436] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:29:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:54.652509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886abe7-81ee-44a7-9c96-30c14bbf71f6 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:54.652509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886abe7-81ee-44a7-9c96-30c14bbf71f6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:54.652509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886abe7-81ee-44a7-9c96-30c14bbf71f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:54.668000 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-4886abe7-81ee-44a7-9c96-30c14bbf71f6 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:29:54.669228 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4886abe7-81ee-44a7-9c96-30c14bbf71f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:29:54.677010 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:54.726790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 224/437] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:54.727937 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0792e-0b4d-4cc7-af98-93ae4558da37 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:54.741492 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 214/438] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:29:54.742371 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886abe7-81ee-44a7-9c96-30c14bbf71f6 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:54.765998 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:54.852770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0792e-0b4d-4cc7-af98-93ae4558da37 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:54.878201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886abe7-81ee-44a7-9c96-30c14bbf71f6 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:54.897111 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65074adf-c95f-4cd1-8c84-1d797ba5f626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:54.897988 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65074adf-c95f-4cd1-8c84-1d797ba5f626 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:54.898440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65074adf-c95f-4cd1-8c84-1d797ba5f626 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:54.898756 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65074adf-c95f-4cd1-8c84-1d797ba5f626 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:54.919376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18717cc-4e24-44f6-a309-6dc85b8644cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:54.920523 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18717cc-4e24-44f6-a309-6dc85b8644cd None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:54.921105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18717cc-4e24-44f6-a309-6dc85b8644cd None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:54.921485 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18717cc-4e24-44f6-a309-6dc85b8644cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:54.938168 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:54.985401 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65074adf-c95f-4cd1-8c84-1d797ba5f626 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:54.987802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65074adf-c95f-4cd1-8c84-1d797ba5f626 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:55.004625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18717cc-4e24-44f6-a309-6dc85b8644cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:55.006282 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18717cc-4e24-44f6-a309-6dc85b8644cd None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:55.027945 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:55.099205 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 225/439] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:55.100070 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65074adf-c95f-4cd1-8c84-1d797ba5f626 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:55.114094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 215/440] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:55.114754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18717cc-4e24-44f6-a309-6dc85b8644cd None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:55.157932 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:55.208249 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65074adf-c95f-4cd1-8c84-1d797ba5f626 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:55.217703 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18717cc-4e24-44f6-a309-6dc85b8644cd None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:55.247850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9c480-40f7-41e2-ad17-d06edcf1d2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:55.248833 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9c480-40f7-41e2-ad17-d06edcf1d2ea None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:55.249323 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9c480-40f7-41e2-ad17-d06edcf1d2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:55.249773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9c480-40f7-41e2-ad17-d06edcf1d2ea None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:55.251490 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9c480-40f7-41e2-ad17-d06edcf1d2ea None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:55.253994 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9c480-40f7-41e2-ad17-d06edcf1d2ea None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:55.256858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 226/441] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:55 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:55.257735 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9c480-40f7-41e2-ad17-d06edcf1d2ea None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:55.263956 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4701a-7730-455b-8135-62467f74764a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:55.264652 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4701a-7730-455b-8135-62467f74764a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:55.264999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:55.265106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4701a-7730-455b-8135-62467f74764a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:55.265526 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4701a-7730-455b-8135-62467f74764a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:55.277950 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4701a-7730-455b-8135-62467f74764a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:29:55.283646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b3b4701a-7730-455b-8135-62467f74764a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:29:55.283646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:29:55.283646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:29:55.283646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:29:55.283646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:29:55.283646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:29:55.283646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:29:55.283646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:29:55.283646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:29:55.283646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:29:55.283646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:29:55.283646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:29:55.283646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:29:55.283646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:29:55.283646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:29:55.283646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:29:55.284538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:29:55.284538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:29:55.284538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:29:55.284538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:29:55.284538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:29:55.284538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:29:55.284538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:29:55.284538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:29:55.284538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:29:55.284538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:29:55.284538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:29:55.284538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:29:55.284538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:29:55.284538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:29:55.284538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:29:55.284538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:29:55.285409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:29:55.285409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:29:55.285409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:29:55.285409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:29:55.285409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:29:55.285409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:29:55.285409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:29:55.285409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:29:55.285409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:29:55.285409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:29:55.285409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:29:55.285409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:29:55.285409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 216/442] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:29:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:29:55.286074 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4701a-7730-455b-8135-62467f74764a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:55.378534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9c480-40f7-41e2-ad17-d06edcf1d2ea None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:55.394215 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4701a-7730-455b-8135-62467f74764a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:55.395900 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:55.416645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0202f5-1a76-4196-829f-6d91323ae8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:55.417416 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0202f5-1a76-4196-829f-6d91323ae8eb None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:55.417943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0202f5-1a76-4196-829f-6d91323ae8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:55.418373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0202f5-1a76-4196-829f-6d91323ae8eb None admin] PATH_INFO: `/v3/projects/4e48d801558a426b8b994cb2bab00a20` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:55.419853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0202f5-1a76-4196-829f-6d91323ae8eb None admin] RBAC: Authorizing `identity:get_project(project_id=4e48d801558a426b8b994cb2bab00a20)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:55.421491 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0202f5-1a76-4196-829f-6d91323ae8eb None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:55.423260 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 227/443] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:55 2022] GET /identity/v3/projects/4e48d801558a426b8b994cb2bab00a20 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:55.424048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0202f5-1a76-4196-829f-6d91323ae8eb None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:55.439241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334f1f5-eea2-49c8-9295-29e2bd9145b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:55.440230 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334f1f5-eea2-49c8-9295-29e2bd9145b8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:55.440750 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334f1f5-eea2-49c8-9295-29e2bd9145b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:55.441092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334f1f5-eea2-49c8-9295-29e2bd9145b8 None admin] PATH_INFO: `/v3/projects/b13122687f1640669b462d728fc006ce` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:55.442908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334f1f5-eea2-49c8-9295-29e2bd9145b8 None admin] RBAC: Authorizing `identity:get_project(project_id=b13122687f1640669b462d728fc006ce)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:55.445461 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334f1f5-eea2-49c8-9295-29e2bd9145b8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:55.448166 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 217/444] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:55 2022] GET /identity/v3/projects/b13122687f1640669b462d728fc006ce => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:55.449474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334f1f5-eea2-49c8-9295-29e2bd9145b8 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:55.519495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:55.551726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0202f5-1a76-4196-829f-6d91323ae8eb None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:55.569026 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334f1f5-eea2-49c8-9295-29e2bd9145b8 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:55.586436 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d91b9-c3f3-4437-9565-584419264a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:55.587235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d91b9-c3f3-4437-9565-584419264a0e None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:55.587836 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d91b9-c3f3-4437-9565-584419264a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:55.588093 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d91b9-c3f3-4437-9565-584419264a0e None admin] PATH_INFO: `/v3/groups/8763f0cf0bf74aa8a2f27c494160c014` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:55.589303 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d91b9-c3f3-4437-9565-584419264a0e None admin] RBAC: Authorizing `identity:get_group(group_id=8763f0cf0bf74aa8a2f27c494160c014)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:55.590899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d91b9-c3f3-4437-9565-584419264a0e None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:55.592538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 228/445] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 26 05:29:55 2022] GET /identity/v3/groups/8763f0cf0bf74aa8a2f27c494160c014 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:55.593357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d91b9-c3f3-4437-9565-584419264a0e None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:55.614877 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93375791-daa5-4921-81aa-64c9d7383c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:55.615773 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93375791-daa5-4921-81aa-64c9d7383c8d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:55.616204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93375791-daa5-4921-81aa-64c9d7383c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:55.616655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93375791-daa5-4921-81aa-64c9d7383c8d None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:55.617823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93375791-daa5-4921-81aa-64c9d7383c8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:55.618261 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93375791-daa5-4921-81aa-64c9d7383c8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:55.620421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93375791-daa5-4921-81aa-64c9d7383c8d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:55.634852 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 218/446] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:29:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:55.635957 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93375791-daa5-4921-81aa-64c9d7383c8d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:55.673268 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:55.729365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d91b9-c3f3-4437-9565-584419264a0e None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:55.754452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93375791-daa5-4921-81aa-64c9d7383c8d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:55.766172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c67a1-f03b-48dd-92cb-93c752b9c603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:55.766823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c67a1-f03b-48dd-92cb-93c752b9c603 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:55.767102 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c67a1-f03b-48dd-92cb-93c752b9c603 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:55.767379 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659c67a1-f03b-48dd-92cb-93c752b9c603 None admin] PATH_INFO: `/v3/groups/8763f0cf0bf74aa8a2f27c494160c014` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:55.768579 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c67a1-f03b-48dd-92cb-93c752b9c603 None admin] RBAC: Authorizing `identity:get_group(group_id=8763f0cf0bf74aa8a2f27c494160c014)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:55.770094 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659c67a1-f03b-48dd-92cb-93c752b9c603 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:55.771764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 229/447] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 26 05:29:55 2022] GET /identity/v3/groups/8763f0cf0bf74aa8a2f27c494160c014 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:55.772580 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c67a1-f03b-48dd-92cb-93c752b9c603 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:55.787514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:55.794513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385e0e5-9427-4ce3-b8bd-ac4df79cb179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:55.795187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385e0e5-9427-4ce3-b8bd-ac4df79cb179 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:55.795443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385e0e5-9427-4ce3-b8bd-ac4df79cb179 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:55.795713 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385e0e5-9427-4ce3-b8bd-ac4df79cb179 None admin] PATH_INFO: `/v3/groups/8763f0cf0bf74aa8a2f27c494160c014` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:55.797082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385e0e5-9427-4ce3-b8bd-ac4df79cb179 None admin] RBAC: Authorizing `identity:get_group(group_id=8763f0cf0bf74aa8a2f27c494160c014)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:55.798601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385e0e5-9427-4ce3-b8bd-ac4df79cb179 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:55.800259 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 219/448] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 26 05:29:55 2022] GET /identity/v3/groups/8763f0cf0bf74aa8a2f27c494160c014 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:55.801048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385e0e5-9427-4ce3-b8bd-ac4df79cb179 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:55.893733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659c67a1-f03b-48dd-92cb-93c752b9c603 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:55.918032 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:55.921486 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385e0e5-9427-4ce3-b8bd-ac4df79cb179 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:55.947225 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41134895-4180-4781-a2ef-bfcbe24fc94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:55.948151 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41134895-4180-4781-a2ef-bfcbe24fc94b None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:55.948642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41134895-4180-4781-a2ef-bfcbe24fc94b None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:55.949224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41134895-4180-4781-a2ef-bfcbe24fc94b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:55.950585 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41134895-4180-4781-a2ef-bfcbe24fc94b None admin] RBAC: Adding query filter params (group.id=8763f0cf0bf74aa8a2f27c494160c014, role.id=bb454faf5dd643b892c6f9b670ad1f2d, scope.project.id=b13122687f1640669b462d728fc006ce) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:55.951135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41134895-4180-4781-a2ef-bfcbe24fc94b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:55.953791 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41134895-4180-4781-a2ef-bfcbe24fc94b None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:55.957926 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebfeb3-6cbd-4490-80b3-5d1e077776d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:55.958553 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebfeb3-6cbd-4490-80b3-5d1e077776d9 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:55.958838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebfeb3-6cbd-4490-80b3-5d1e077776d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:55.959109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebfeb3-6cbd-4490-80b3-5d1e077776d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:55.974038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 230/449] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 26 05:29:55 2022] GET /identity/v3/role_assignments?group.id=8763f0cf0bf74aa8a2f27c494160c014&scope.project.id=b13122687f1640669b462d728fc006ce&role.id=bb454faf5dd643b892c6f9b670ad1f2d => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:55.974038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41134895-4180-4781-a2ef-bfcbe24fc94b None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:56.049447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ebfeb3-6cbd-4490-80b3-5d1e077776d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:56.051045 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ebfeb3-6cbd-4490-80b3-5d1e077776d9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:56.070942 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:56.126264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41134895-4180-4781-a2ef-bfcbe24fc94b None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:56.172991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc2b0e-502b-495a-9909-5b6416524111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:56.173662 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc2b0e-502b-495a-9909-5b6416524111 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:56.173945 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc2b0e-502b-495a-9909-5b6416524111 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:56.174216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc2b0e-502b-495a-9909-5b6416524111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:56.175621 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc2b0e-502b-495a-9909-5b6416524111 None admin] RBAC: Adding query filter params (group.id=8763f0cf0bf74aa8a2f27c494160c014, role.id=bb454faf5dd643b892c6f9b670ad1f2d, scope.project.id=4e48d801558a426b8b994cb2bab00a20) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:56.175948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc2b0e-502b-495a-9909-5b6416524111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:56.177919 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc2b0e-502b-495a-9909-5b6416524111 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:56.186620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 231/450] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 26 05:29:55 2022] GET /identity/v3/role_assignments?group.id=8763f0cf0bf74aa8a2f27c494160c014&scope.project.id=4e48d801558a426b8b994cb2bab00a20&role.id=bb454faf5dd643b892c6f9b670ad1f2d => generated 611 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:56.187397 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc2b0e-502b-495a-9909-5b6416524111 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:56.200525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:56.209179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 220/451] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:29:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:29:56.216303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebfeb3-6cbd-4490-80b3-5d1e077776d9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:56.339158 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc2b0e-502b-495a-9909-5b6416524111 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:56.344544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebfeb3-6cbd-4490-80b3-5d1e077776d9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:56.383616 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cf5f0-2089-4f6d-894e-fd5442a3b56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:56.384450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cf5f0-2089-4f6d-894e-fd5442a3b56f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:56.384596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fb367-33ae-4ec1-a919-6774c0324147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:56.384790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cf5f0-2089-4f6d-894e-fd5442a3b56f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:56.385006 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cf5f0-2089-4f6d-894e-fd5442a3b56f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:56.385283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fb367-33ae-4ec1-a919-6774c0324147 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:56.385561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fb367-33ae-4ec1-a919-6774c0324147 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:56.385817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851cf5f0-2089-4f6d-894e-fd5442a3b56f None admin] RBAC: Adding query filter params (group.id=8763f0cf0bf74aa8a2f27c494160c014, role.id=05e78a4a009c41488cf3d2c47e8cbe30, scope.project.id=b13122687f1640669b462d728fc006ce) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:56.385912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fb367-33ae-4ec1-a919-6774c0324147 None admin] PATH_INFO: `/v3/projects/fb1661fcfa594d16a5b5329ef20333a6` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:56.386131 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851cf5f0-2089-4f6d-894e-fd5442a3b56f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:56.387196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308fb367-33ae-4ec1-a919-6774c0324147 None admin] RBAC: Authorizing `identity:get_project(project_id=fb1661fcfa594d16a5b5329ef20333a6)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:56.387742 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851cf5f0-2089-4f6d-894e-fd5442a3b56f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:56.388667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308fb367-33ae-4ec1-a919-6774c0324147 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:56.390838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 221/452] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:29:56 2022] GET /identity/v3/projects/fb1661fcfa594d16a5b5329ef20333a6 => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:56.391801 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:56.396581 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fb367-33ae-4ec1-a919-6774c0324147 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:56.397185 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 232/453] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 26 05:29:56 2022] GET /identity/v3/role_assignments?group.id=8763f0cf0bf74aa8a2f27c494160c014&scope.project.id=b13122687f1640669b462d728fc006ce&role.id=05e78a4a009c41488cf3d2c47e8cbe30 => generated 611 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:29:56.526908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:29:56.590158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fb367-33ae-4ec1-a919-6774c0324147 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:56.653825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7f5a7-df3e-4846-9b97-65ac79811e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:56.655177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7f5a7-df3e-4846-9b97-65ac79811e78 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:56.655896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7f5a7-df3e-4846-9b97-65ac79811e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:56.656361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7f5a7-df3e-4846-9b97-65ac79811e78 None admin] PATH_INFO: `/v3/groups/865cbf2df60b4542b47017e1e0c5737e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:56.658474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7f5a7-df3e-4846-9b97-65ac79811e78 None admin] RBAC: Authorizing `identity:get_group(group_id=865cbf2df60b4542b47017e1e0c5737e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:56.661099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7f5a7-df3e-4846-9b97-65ac79811e78 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:56.664643 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 222/454] 10.222.0.57 () {56 vars in 1180 bytes} [Mon Sep 26 05:29:56 2022] GET /identity/v3/groups/865cbf2df60b4542b47017e1e0c5737e => generated 229 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:29:56.670600 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cf5f0-2089-4f6d-894e-fd5442a3b56f None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:29:56.811496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cf5f0-2089-4f6d-894e-fd5442a3b56f None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:29:56.853657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab7820-1343-4edb-989e-a1121c28ae3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:29:56.854282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab7820-1343-4edb-989e-a1121c28ae3d None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:29:56.854547 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab7820-1343-4edb-989e-a1121c28ae3d None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:29:56.854826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab7820-1343-4edb-989e-a1121c28ae3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:29:56.855567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab7820-1343-4edb-989e-a1121c28ae3d None admin] RBAC: Adding query filter params (group.id=865cbf2df60b4542b47017e1e0c5737e, role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=fb1661fcfa594d16a5b5329ef20333a6) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:29:56.855870 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab7820-1343-4edb-989e-a1121c28ae3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:29:56.857413 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab7820-1343-4edb-989e-a1121c28ae3d None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:29:56.864566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 233/455] 10.222.0.57 () {56 vars in 1376 bytes} [Mon Sep 26 05:29:56 2022] GET /identity/v3/role_assignments?group.id=865cbf2df60b4542b47017e1e0c5737e&scope.project.id=fb1661fcfa594d16a5b5329ef20333a6&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:30:00.378875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983e4f2-0fd0-4c59-9cd3-92b8674ebfc9 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:00.378875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983e4f2-0fd0-4c59-9cd3-92b8674ebfc9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:00.378875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983e4f2-0fd0-4c59-9cd3-92b8674ebfc9 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:00.380322 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 223/456] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:00.389577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b778bf-5745-4ee0-947d-00f6b751927a None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:00.390903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b778bf-5745-4ee0-947d-00f6b751927a None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:00.391644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b778bf-5745-4ee0-947d-00f6b751927a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:00.425732 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-f2b778bf-5745-4ee0-947d-00f6b751927a None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:00.428215 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f2b778bf-5745-4ee0-947d-00f6b751927a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:00.533625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:00.551886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 234/457] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:00.747725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d816b-7764-4e58-93b3-b93f5e72ed67 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:00.748805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d816b-7764-4e58-93b3-b93f5e72ed67 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:00.749615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d816b-7764-4e58-93b3-b93f5e72ed67 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:00.752977 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 224/458] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:00.759597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c3229-2bb4-44b8-8bd5-28b13c1251b3 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:00.760572 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c3229-2bb4-44b8-8bd5-28b13c1251b3 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:00.761394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c3229-2bb4-44b8-8bd5-28b13c1251b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:00.802927 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-893c3229-2bb4-44b8-8bd5-28b13c1251b3 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:00.805765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-893c3229-2bb4-44b8-8bd5-28b13c1251b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:00.843160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caef6f2-2b9c-45ae-ae46-31253cb2cb54 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:00.843808 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caef6f2-2b9c-45ae-ae46-31253cb2cb54 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:00.844296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caef6f2-2b9c-45ae-ae46-31253cb2cb54 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:00.847966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 225/459] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:00.853948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcc8ab-60b2-45a0-9aa5-a7b7f5a37c37 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:00.854447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcc8ab-60b2-45a0-9aa5-a7b7f5a37c37 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:00.854943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcc8ab-60b2-45a0-9aa5-a7b7f5a37c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:00.890381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-81bcc8ab-60b2-45a0-9aa5-a7b7f5a37c37 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:00.891702 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-81bcc8ab-60b2-45a0-9aa5-a7b7f5a37c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:00.923863 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 235/460] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:30:01.018747 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 226/461] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:01.071471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c3229-2bb4-44b8-8bd5-28b13c1251b3 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.221524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c3229-2bb4-44b8-8bd5-28b13c1251b3 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.240194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcc8ab-60b2-45a0-9aa5-a7b7f5a37c37 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.273566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:01.293523 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11a1e0-2680-486d-b45d-4286cd05c0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.294178 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11a1e0-2680-486d-b45d-4286cd05c0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.295400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11a1e0-2680-486d-b45d-4286cd05c0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.295846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11a1e0-2680-486d-b45d-4286cd05c0d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.319845 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11a1e0-2680-486d-b45d-4286cd05c0d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.321662 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11a1e0-2680-486d-b45d-4286cd05c0d9 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.341853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-5c11a1e0-2680-486d-b45d-4286cd05c0d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:01.341853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:01.341853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:01.341853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:01.341853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:01.341853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:01.341853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:01.341853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:01.341853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:01.341853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:01.341853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:01.341853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:01.341853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:01.341853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:01.341853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:01.341853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:01.342981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:01.342981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:01.342981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:01.342981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:01.342981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:01.342981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:01.342981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:01.342981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:01.342981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:01.342981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:01.342981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:01.342981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:01.342981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:01.342981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:01.342981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:01.342981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:01.342981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:01.343963 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:01.343963 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:01.343963 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:01.345228 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 236/462] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:01.346252 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11a1e0-2680-486d-b45d-4286cd05c0d9 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.360054 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:01.417933 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcc8ab-60b2-45a0-9aa5-a7b7f5a37c37 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.466648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11a1e0-2680-486d-b45d-4286cd05c0d9 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.468782 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b300d3-a0e0-4b41-bf9e-b38249a7c6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.469840 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b300d3-a0e0-4b41-bf9e-b38249a7c6ab None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.471197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b300d3-a0e0-4b41-bf9e-b38249a7c6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.471563 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b300d3-a0e0-4b41-bf9e-b38249a7c6ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.486863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b300d3-a0e0-4b41-bf9e-b38249a7c6ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.489266 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b300d3-a0e0-4b41-bf9e-b38249a7c6ab None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.507593 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cb25a-8889-46bd-91fb-214da5db5e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.508541 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cb25a-8889-46bd-91fb-214da5db5e87 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.509017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cb25a-8889-46bd-91fb-214da5db5e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.509487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10cb25a-8889-46bd-91fb-214da5db5e87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.510404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-d9b300d3-a0e0-4b41-bf9e-b38249a7c6ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:01.510404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:01.510404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:01.510404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:01.510404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:01.510404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:01.510404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:01.510404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:01.510404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:01.510404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:01.510404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:01.510404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:01.510404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:01.510404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:01.510404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:01.510404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:01.511280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:01.511280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:01.511280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:01.511280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:01.511280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:01.511280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:01.511280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:01.511280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:01.511280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:01.511280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:01.511280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:01.511280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:01.511280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:01.511280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:01.511280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:01.511280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:01.511280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:01.512188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:01.512188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:01.512188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:01.512188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 227/463] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:01.512792 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b300d3-a0e0-4b41-bf9e-b38249a7c6ab None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.529994 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10cb25a-8889-46bd-91fb-214da5db5e87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.531471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10cb25a-8889-46bd-91fb-214da5db5e87 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.547752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-e10cb25a-8889-46bd-91fb-214da5db5e87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:01.547752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:01.547752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:01.547752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:01.547752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:01.547752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:01.547752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:01.547752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:01.547752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:01.547752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:01.547752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:01.547752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:01.547752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:01.547752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:01.547752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:01.547752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:01.548747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:01.548747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:01.548747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:01.548747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:01.548747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:01.548747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:01.548747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:01.548747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:01.548747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:01.548747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:01.548747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:01.548747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:01.548747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:01.548747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:01.548747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:01.548747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:01.548747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:01.549638 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:01.549638 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:01.549638 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:01.554243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 237/464] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:01.555333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cb25a-8889-46bd-91fb-214da5db5e87 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.587362 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:01.650379 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b300d3-a0e0-4b41-bf9e-b38249a7c6ab None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.664261 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10cb25a-8889-46bd-91fb-214da5db5e87 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.670660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:01.691620 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0518a3-00ab-4dc7-8ef2-da37c00a08f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.692689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0518a3-00ab-4dc7-8ef2-da37c00a08f7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.693268 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0518a3-00ab-4dc7-8ef2-da37c00a08f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.693745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0518a3-00ab-4dc7-8ef2-da37c00a08f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.694638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0518a3-00ab-4dc7-8ef2-da37c00a08f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:01.695457 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0518a3-00ab-4dc7-8ef2-da37c00a08f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.697658 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0518a3-00ab-4dc7-8ef2-da37c00a08f7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.701341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8148d-3f15-4a82-be54-39cc6e99432c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.702195 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8148d-3f15-4a82-be54-39cc6e99432c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.702617 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8148d-3f15-4a82-be54-39cc6e99432c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.703027 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8148d-3f15-4a82-be54-39cc6e99432c None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.703878 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8148d-3f15-4a82-be54-39cc6e99432c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:01.704320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8148d-3f15-4a82-be54-39cc6e99432c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.706051 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8148d-3f15-4a82-be54-39cc6e99432c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.717304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 228/465] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:01.718538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0518a3-00ab-4dc7-8ef2-da37c00a08f7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.721195 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 238/466] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:01.726487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8148d-3f15-4a82-be54-39cc6e99432c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.826372 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:01.841495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0518a3-00ab-4dc7-8ef2-da37c00a08f7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.841686 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8148d-3f15-4a82-be54-39cc6e99432c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:01.883979 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194143c2-2eb9-4ff4-aef5-94b5bdc52df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.884625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194143c2-2eb9-4ff4-aef5-94b5bdc52df4 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.885121 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194143c2-2eb9-4ff4-aef5-94b5bdc52df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.885674 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194143c2-2eb9-4ff4-aef5-94b5bdc52df4 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.886584 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194143c2-2eb9-4ff4-aef5-94b5bdc52df4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.888481 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194143c2-2eb9-4ff4-aef5-94b5bdc52df4 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.889124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8efeb-9ab0-4908-b253-92b28757af8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:01.889879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8efeb-9ab0-4908-b253-92b28757af8a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:01.890254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8efeb-9ab0-4908-b253-92b28757af8a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:01.890623 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8efeb-9ab0-4908-b253-92b28757af8a None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:01.891375 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8efeb-9ab0-4908-b253-92b28757af8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:01.891777 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8efeb-9ab0-4908-b253-92b28757af8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:01.893319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8efeb-9ab0-4908-b253-92b28757af8a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:01.909249 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 229/467] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:01.909249 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8efeb-9ab0-4908-b253-92b28757af8a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.925176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 239/468] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:30:01 2022] POST /identity/v3/users => generated 246 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:01.925176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194143c2-2eb9-4ff4-aef5-94b5bdc52df4 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:01.969621 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:02.046899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8efeb-9ab0-4908-b253-92b28757af8a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:02.070801 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194143c2-2eb9-4ff4-aef5-94b5bdc52df4 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:02.087006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116546b9-81c8-4384-89fe-d8e45b1c596b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:02.087656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116546b9-81c8-4384-89fe-d8e45b1c596b None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:02.087968 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116546b9-81c8-4384-89fe-d8e45b1c596b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:02.088189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116546b9-81c8-4384-89fe-d8e45b1c596b None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:02.088865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116546b9-81c8-4384-89fe-d8e45b1c596b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:02.090476 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116546b9-81c8-4384-89fe-d8e45b1c596b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:02.115789 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6ef6f-def5-464b-a968-9147174c2018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:02.116328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6ef6f-def5-464b-a968-9147174c2018 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:02.116638 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6ef6f-def5-464b-a968-9147174c2018 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:02.116915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6ef6f-def5-464b-a968-9147174c2018 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:02.117639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d6ef6f-def5-464b-a968-9147174c2018 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:02.125546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d6ef6f-def5-464b-a968-9147174c2018 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:02.125799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 230/469] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:30:01 2022] POST /identity/v3/users => generated 248 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:02.125799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d38c15-7ae3-418b-a34f-ab4deaa83b3d None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:02.125799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d38c15-7ae3-418b-a34f-ab4deaa83b3d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:02.125799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d38c15-7ae3-418b-a34f-ab4deaa83b3d None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:02.125799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 231/470] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:02.125799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa2245-b1d7-4915-a108-3ef5f3175fa7 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:02.125799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa2245-b1d7-4915-a108-3ef5f3175fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:02.126551 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa2245-b1d7-4915-a108-3ef5f3175fa7 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:02.126551 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 232/471] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:02.131759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd8df7-958e-4692-a93e-2999501906db None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:02.131759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd8df7-958e-4692-a93e-2999501906db None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:02.131759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd8df7-958e-4692-a93e-2999501906db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:02.154028 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-54dd8df7-958e-4692-a93e-2999501906db None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:02.155704 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:02.156355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-54dd8df7-958e-4692-a93e-2999501906db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:02.171411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 240/472] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:30:01 2022] POST /identity/v3/users => generated 248 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:02.174836 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7949e-1882-4772-8ae9-78b53b5c1367 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:02.175766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7949e-1882-4772-8ae9-78b53b5c1367 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:02.176586 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7949e-1882-4772-8ae9-78b53b5c1367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:02.217320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-7fb7949e-1882-4772-8ae9-78b53b5c1367 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:02.221344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7fb7949e-1882-4772-8ae9-78b53b5c1367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:02.280571 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 233/473] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:02.383399 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 241/474] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:02.562870 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd8df7-958e-4692-a93e-2999501906db None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:02.590532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:02.679612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd8df7-958e-4692-a93e-2999501906db None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:02.729785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a28b37-03d2-4ba1-99cf-cd6695ffbf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:02.730979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a28b37-03d2-4ba1-99cf-cd6695ffbf41 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:02.731457 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a28b37-03d2-4ba1-99cf-cd6695ffbf41 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:02.731935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a28b37-03d2-4ba1-99cf-cd6695ffbf41 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:02.733189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a28b37-03d2-4ba1-99cf-cd6695ffbf41 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:02.735753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a28b37-03d2-4ba1-99cf-cd6695ffbf41 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:02.761953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.sql.core [None req-17a28b37-03d2-4ba1-99cf-cd6695ffbf41 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 26 05:30:02.761953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 26 05:30:02.761953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [parameters: {'id': 'bd1a89b796944bdab595ec2e71b2cea1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 26 05:30:02.761953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81664) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 26 05:30:02.794395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-17a28b37-03d2-4ba1-99cf-cd6695ffbf41 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 26 05:30:02.794395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:02.794395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 05:30:02.794395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 05:30:02.794395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 05:30:02.794395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 05:30:02.794395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 05:30:02.794395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 05:30:02.794395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 05:30:02.794395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application conn.query(q) Sep 26 05:30:02.794395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 05:30:02.794395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 05:30:02.794395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 05:30:02.794395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application result.read() Sep 26 05:30:02.794395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return ref.to_dict() Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 05:30:02.795394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application next(self.gen) Sep 26 05:30:02.797167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 26 05:30:02.797167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application yield resource Sep 26 05:30:02.797167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 05:30:02.797167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application next(self.gen) Sep 26 05:30:02.797167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 26 05:30:02.797167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self.session.rollback() Sep 26 05:30:02.797167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 26 05:30:02.797167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self.force_reraise() Sep 26 05:30:02.797167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 26 05:30:02.797167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise self.value Sep 26 05:30:02.797167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 26 05:30:02.797167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 26 05:30:02.797167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 26 05:30:02.797167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application session.commit() Sep 26 05:30:02.797167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 26 05:30:02.797167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 26 05:30:02.797167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 26 05:30:02.798092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._prepare_impl() Sep 26 05:30:02.798092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 26 05:30:02.798092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self.session.flush() Sep 26 05:30:02.798092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 26 05:30:02.798092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._flush(objects) Sep 26 05:30:02.798092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 26 05:30:02.798092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 26 05:30:02.798092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 26 05:30:02.798092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application compat.raise_( Sep 26 05:30:02.798092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 05:30:02.798092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception Sep 26 05:30:02.798092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 26 05:30:02.798092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application flush_context.execute() Sep 26 05:30:02.798092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 26 05:30:02.798092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rec.execute(self) Sep 26 05:30:02.798092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 26 05:30:02.798092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 26 05:30:02.799019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 26 05:30:02.799019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 26 05:30:02.799019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 26 05:30:02.799019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 26 05:30:02.799019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 26 05:30:02.799019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 26 05:30:02.799019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 26 05:30:02.799019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 26 05:30:02.799019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 26 05:30:02.799019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 26 05:30:02.799019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 26 05:30:02.799019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 26 05:30:02.799019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 26 05:30:02.799019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 26 05:30:02.799019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 05:30:02.799019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception Sep 26 05:30:02.801101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 05:30:02.801101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 05:30:02.801101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 05:30:02.801101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 05:30:02.801101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 05:30:02.801101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 05:30:02.801101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 05:30:02.801101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application conn.query(q) Sep 26 05:30:02.801101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 05:30:02.801101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 05:30:02.801101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 05:30:02.801101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application result.read() Sep 26 05:30:02.801101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 05:30:02.801101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 05:30:02.801101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 05:30:02.801101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 05:30:02.803059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 05:30:02.803059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 05:30:02.803059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 05:30:02.803059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 05:30:02.803059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 26 05:30:02.803059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 26 05:30:02.803059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [parameters: {'id': 'bd1a89b796944bdab595ec2e71b2cea1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 26 05:30:02.803059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 26 05:30:02.803059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:02.803059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:30:02.803059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:02.803059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:02.803059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:02.803059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:02.803059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:02.803059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:02.805062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:02.805062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:02.805062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:02.805062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:02.805062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:02.805062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:02.805062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 26 05:30:02.805062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 26 05:30:02.805062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:02.805062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:02.805062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 26 05:30:02.805062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 26 05:30:02.805062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 26 05:30:02.805062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 26 05:30:02.805062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 26 05:30:02.805062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:02.808196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 234/475] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Sep 26 05:30:02 2022] POST /identity/v3/roles => generated 144 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:02.815680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7949e-1882-4772-8ae9-78b53b5c1367 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:02.872106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a28b37-03d2-4ba1-99cf-cd6695ffbf41 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:03.022924 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:03.061541 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a28b37-03d2-4ba1-99cf-cd6695ffbf41 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:03.107153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7949e-1882-4772-8ae9-78b53b5c1367 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:03.188581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:03.198285 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7acdb-1aa5-49d0-8853-f020392b07f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.199401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7acdb-1aa5-49d0-8853-f020392b07f0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.199834 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04e5cc-f26f-4e79-ac3e-7480917250dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.200044 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7acdb-1aa5-49d0-8853-f020392b07f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.200302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7acdb-1aa5-49d0-8853-f020392b07f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.201733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04e5cc-f26f-4e79-ac3e-7480917250dd None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.202484 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04e5cc-f26f-4e79-ac3e-7480917250dd None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.203176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04e5cc-f26f-4e79-ac3e-7480917250dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.238343 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04e5cc-f26f-4e79-ac3e-7480917250dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:03.249324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-bc04e5cc-f26f-4e79-ac3e-7480917250dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:30:03.249324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:03.249324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:03.249324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:03.249324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:03.249324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:03.249324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:03.249324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:03.249324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:03.249324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:03.249324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:03.249324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:03.249324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:03.249324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:03.249324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:03.249324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:03.251089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:03.251089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:03.251089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:03.251089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:03.251089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:03.251089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:03.251089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:03.251089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:03.251089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:03.251089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:03.251089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:03.251089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:03.251089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:03.251089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:03.251089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:03.251089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:03.252671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:03.252671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:03.252671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:03.252671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:03.252671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:03.252671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:03.252671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:03.252671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:03.252671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:03.252671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:03.252671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 05:30:03.252671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:03.257116 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 242/476] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:30:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 443 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:03.258538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca7acdb-1aa5-49d0-8853-f020392b07f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:03.265385 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca7acdb-1aa5-49d0-8853-f020392b07f0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:03.265613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04e5cc-f26f-4e79-ac3e-7480917250dd None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:03.314255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-dca7acdb-1aa5-49d0-8853-f020392b07f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:03.314255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:03.314255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:03.314255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:03.314255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:03.314255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:03.314255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:03.314255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:03.314255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:03.314255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:03.314255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:03.314255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:03.314255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:03.314255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:03.314255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:03.314255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:03.316304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:03.316304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:03.316304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:03.316304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:03.316304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:03.316304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:03.316304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:03.316304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:03.316304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:03.316304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:03.316304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:03.316304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:03.316304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:03.316304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:03.316304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:03.316304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:03.316304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:03.318248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:03.318248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:03.318248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:03.327350 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 235/477] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 456 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:03.333296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7acdb-1aa5-49d0-8853-f020392b07f0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:03.461182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04e5cc-f26f-4e79-ac3e-7480917250dd None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:03.493344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7301e5-2816-4214-a510-92fe85f2b3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.493964 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7301e5-2816-4214-a510-92fe85f2b3de None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.494239 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7301e5-2816-4214-a510-92fe85f2b3de None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.494507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7301e5-2816-4214-a510-92fe85f2b3de None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.495177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7301e5-2816-4214-a510-92fe85f2b3de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:03.495478 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7301e5-2816-4214-a510-92fe85f2b3de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:03.497153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7301e5-2816-4214-a510-92fe85f2b3de None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:03.512634 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 243/478] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:03.622160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7acdb-1aa5-49d0-8853-f020392b07f0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:03.686059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e83252-9ad6-4fda-a4d2-3cfc673b7888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.686707 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e83252-9ad6-4fda-a4d2-3cfc673b7888 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.686989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e83252-9ad6-4fda-a4d2-3cfc673b7888 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.687266 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e83252-9ad6-4fda-a4d2-3cfc673b7888 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.687942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e83252-9ad6-4fda-a4d2-3cfc673b7888 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:03.688323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e83252-9ad6-4fda-a4d2-3cfc673b7888 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:03.689809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e83252-9ad6-4fda-a4d2-3cfc673b7888 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:03.701973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:03.708739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 236/479] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:03.714113 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7301e5-2816-4214-a510-92fe85f2b3de None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:03.799682 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:03.905938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7301e5-2816-4214-a510-92fe85f2b3de None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:03.972375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fda84c-b06d-4030-ad7f-55917cb54bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:03.973416 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fda84c-b06d-4030-ad7f-55917cb54bba None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:03.973898 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fda84c-b06d-4030-ad7f-55917cb54bba None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:03.974364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fda84c-b06d-4030-ad7f-55917cb54bba None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:03.975546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fda84c-b06d-4030-ad7f-55917cb54bba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:03.978270 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fda84c-b06d-4030-ad7f-55917cb54bba None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:04.017786 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 244/480] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:30:03 2022] POST /identity/v3/users => generated 249 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:30:05.529311 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88655e4a-ee55-4f31-97ed-1753ab31af28 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:05.530869 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88655e4a-ee55-4f31-97ed-1753ab31af28 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:05.531651 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88655e4a-ee55-4f31-97ed-1753ab31af28 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:05.534922 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 237/481] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:05.543764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b385b-2ec6-4e74-b7a0-08aeb6313e8d None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:05.544587 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b385b-2ec6-4e74-b7a0-08aeb6313e8d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:05.544718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b385b-2ec6-4e74-b7a0-08aeb6313e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:05.586862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-186b385b-2ec6-4e74-b7a0-08aeb6313e8d None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:05.588809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-186b385b-2ec6-4e74-b7a0-08aeb6313e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:05.747017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 245/482] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:05.960369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241a2c0-de24-49c1-b173-b3c23458c81a None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:05.961446 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241a2c0-de24-49c1-b173-b3c23458c81a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:05.962247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241a2c0-de24-49c1-b173-b3c23458c81a None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:05.965565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 238/483] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:05.974798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246bf77-109c-41bf-85e8-1bcb3dc57107 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:05.975275 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246bf77-109c-41bf-85e8-1bcb3dc57107 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:05.975752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246bf77-109c-41bf-85e8-1bcb3dc57107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.023320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-2246bf77-109c-41bf-85e8-1bcb3dc57107 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:06.025313 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2246bf77-109c-41bf-85e8-1bcb3dc57107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:06.098969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:06.177460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241a2c0-de24-49c1-b173-b3c23458c81a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:06.184897 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 246/484] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:06.258894 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33daf4-bf02-4be0-8fe1-db072669463a None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.259339 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33daf4-bf02-4be0-8fe1-db072669463a None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.259795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33daf4-bf02-4be0-8fe1-db072669463a None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.263056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 247/485] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:30:06.271658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325de543-b594-4cab-a520-a0a929e4de90 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.272104 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325de543-b594-4cab-a520-a0a929e4de90 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.272576 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325de543-b594-4cab-a520-a0a929e4de90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.303718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-325de543-b594-4cab-a520-a0a929e4de90 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:06.305781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-325de543-b594-4cab-a520-a0a929e4de90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:06.417935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241a2c0-de24-49c1-b173-b3c23458c81a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:06.436859 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 248/486] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 05:30:06.491032 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84c421-778b-4979-8b32-cc975d802bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:06.491585 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84c421-778b-4979-8b32-cc975d802bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.491862 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84c421-778b-4979-8b32-cc975d802bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.492132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84c421-778b-4979-8b32-cc975d802bf7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.513151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84c421-778b-4979-8b32-cc975d802bf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:06.518273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-ab84c421-778b-4979-8b32-cc975d802bf7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:06.518273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:06.518273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:06.518273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:06.518273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:06.518273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:06.518273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:06.518273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:06.518273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:06.518273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:06.518273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:06.518273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:06.518273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:06.518273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:06.518273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:06.518273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:06.520007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:06.520007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:06.520007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:06.520007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:06.520007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:06.520007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:06.520007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:06.520007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:06.520007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:06.520007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:06.520007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:06.520007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:06.520007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:06.520007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:06.520007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:06.520007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:06.521631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:06.521631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:06.521631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:06.521631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:06.521631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:06.521631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:06.521631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:06.521631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:06.521631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:06.521631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:06.521631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:06.521631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:06.521631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 239/487] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:30:06.531364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325de543-b594-4cab-a520-a0a929e4de90 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:06.735323 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:06.742243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325de543-b594-4cab-a520-a0a929e4de90 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:06.759886 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84c421-778b-4979-8b32-cc975d802bf7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:06.818423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadb6b2-76ab-4eb0-88ba-17526586f725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:06.819449 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadb6b2-76ab-4eb0-88ba-17526586f725 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:06.819944 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadb6b2-76ab-4eb0-88ba-17526586f725 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:06.820433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadb6b2-76ab-4eb0-88ba-17526586f725 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:06.822243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadb6b2-76ab-4eb0-88ba-17526586f725 None admin] RBAC: Adding query filter params (name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:06.822789 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadb6b2-76ab-4eb0-88ba-17526586f725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:06.825495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadb6b2-76ab-4eb0-88ba-17526586f725 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:06.851155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 249/488] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:30:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:06.852818 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadb6b2-76ab-4eb0-88ba-17526586f725 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:06.924557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:07.006909 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84c421-778b-4979-8b32-cc975d802bf7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:07.060923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadb6b2-76ab-4eb0-88ba-17526586f725 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:07.078341 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d97930-e7f5-45bc-b4ca-11c10e46cf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:07.079510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d97930-e7f5-45bc-b4ca-11c10e46cf1f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:07.080014 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d97930-e7f5-45bc-b4ca-11c10e46cf1f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:07.080708 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d97930-e7f5-45bc-b4ca-11c10e46cf1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:07.103552 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d97930-e7f5-45bc-b4ca-11c10e46cf1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:07.113529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-92d97930-e7f5-45bc-b4ca-11c10e46cf1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:07.113529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:07.113529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:07.113529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:07.113529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:07.113529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:07.113529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:07.113529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:07.113529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:07.113529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:07.113529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:07.113529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:07.113529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:07.113529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:07.113529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:07.113529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:07.114599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:07.114599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:07.114599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:07.114599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:07.114599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:07.114599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:07.114599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:07.114599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:07.114599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:07.114599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:07.114599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:07.114599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:07.114599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:07.114599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:07.114599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:07.114599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:07.115421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:07.115421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:07.115421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:07.115421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:07.115421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:07.115421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:07.115421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:07.115421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:07.115421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:07.115421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:07.115421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:07.115421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:07.116587 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 240/489] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:07.121660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d97930-e7f5-45bc-b4ca-11c10e46cf1f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:07.121986 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b878dd-a1d2-4e6b-aa88-0ccd21a20dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:07.121986 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b878dd-a1d2-4e6b-aa88-0ccd21a20dbd None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:07.122428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b878dd-a1d2-4e6b-aa88-0ccd21a20dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:07.123213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b878dd-a1d2-4e6b-aa88-0ccd21a20dbd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:07.140946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b878dd-a1d2-4e6b-aa88-0ccd21a20dbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:07.151666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-82b878dd-a1d2-4e6b-aa88-0ccd21a20dbd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:07.151666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:07.151666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:07.151666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:07.151666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:07.151666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:07.151666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:07.151666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:07.151666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:07.151666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:07.151666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:07.151666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:07.151666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:07.151666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:07.151666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:07.151666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:07.153748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:07.153748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:07.153748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:07.153748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:07.153748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:07.153748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:07.153748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:07.153748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:07.153748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:07.153748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:07.153748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:07.153748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:07.153748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:07.153748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:07.153748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:07.153748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:07.155506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:07.155506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:07.155506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:07.155506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:07.155506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:07.155506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:07.155506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:07.155506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:07.155506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:07.155506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:07.155506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:07.155506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:07.161819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 250/490] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:07.163047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b878dd-a1d2-4e6b-aa88-0ccd21a20dbd None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:07.208576 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:07.337619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d97930-e7f5-45bc-b4ca-11c10e46cf1f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:07.380137 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:07.409169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b878dd-a1d2-4e6b-aa88-0ccd21a20dbd None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:07.409599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457f69f-e09b-4623-a9ec-062af6bbb383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:07.410916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457f69f-e09b-4623-a9ec-062af6bbb383 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:07.411623 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457f69f-e09b-4623-a9ec-062af6bbb383 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:07.412452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457f69f-e09b-4623-a9ec-062af6bbb383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:07.435683 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457f69f-e09b-4623-a9ec-062af6bbb383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:07.439185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457f69f-e09b-4623-a9ec-062af6bbb383 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:07.470401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-7457f69f-e09b-4623-a9ec-062af6bbb383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:07.470401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:07.470401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:07.470401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:07.470401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:07.470401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:07.470401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:07.470401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:07.470401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:07.470401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:07.470401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:07.470401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:07.470401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:07.470401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:07.470401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:07.470401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:07.472790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:07.472790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:07.472790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:07.472790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:07.472790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:07.472790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:07.472790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:07.472790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:07.472790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:07.472790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:07.472790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:07.472790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:07.472790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:07.472790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:07.472790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:07.472790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:07.472790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:07.474841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:07.474841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:07.474841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:07.476222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 241/491] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:07.477756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457f69f-e09b-4623-a9ec-062af6bbb383 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:07.490953 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9a652-ad58-4351-8ea2-cb0a6e66ec8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:07.490953 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9a652-ad58-4351-8ea2-cb0a6e66ec8e None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:07.490953 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9a652-ad58-4351-8ea2-cb0a6e66ec8e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:07.490953 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9a652-ad58-4351-8ea2-cb0a6e66ec8e None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:07.495208 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9a652-ad58-4351-8ea2-cb0a6e66ec8e None admin] RBAC: Adding query filter params (name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:07.496323 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9a652-ad58-4351-8ea2-cb0a6e66ec8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:07.506569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9a652-ad58-4351-8ea2-cb0a6e66ec8e None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:07.560395 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 251/492] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:30:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:07.562447 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9a652-ad58-4351-8ea2-cb0a6e66ec8e None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:07.794532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457f69f-e09b-4623-a9ec-062af6bbb383 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:07.810393 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:07.838310 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9a652-ad58-4351-8ea2-cb0a6e66ec8e None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:07.855991 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168f44f-785a-4eb0-bdd5-29ce3f996691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:07.857046 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168f44f-785a-4eb0-bdd5-29ce3f996691 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:07.857613 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168f44f-785a-4eb0-bdd5-29ce3f996691 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:07.858126 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168f44f-785a-4eb0-bdd5-29ce3f996691 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:07.859274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168f44f-785a-4eb0-bdd5-29ce3f996691 None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:07.860138 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168f44f-785a-4eb0-bdd5-29ce3f996691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:07.862492 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168f44f-785a-4eb0-bdd5-29ce3f996691 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:07.877548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 242/493] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:30:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:07.882411 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168f44f-785a-4eb0-bdd5-29ce3f996691 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:07.887994 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f0dc1-b70c-48b2-9f4e-c01d3cddb6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:07.888629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f0dc1-b70c-48b2-9f4e-c01d3cddb6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:07.888905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f0dc1-b70c-48b2-9f4e-c01d3cddb6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:07.889210 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f0dc1-b70c-48b2-9f4e-c01d3cddb6f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:07.889942 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f0dc1-b70c-48b2-9f4e-c01d3cddb6f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:07.890447 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f0dc1-b70c-48b2-9f4e-c01d3cddb6f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:07.891864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f0dc1-b70c-48b2-9f4e-c01d3cddb6f6 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:07.908688 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 252/494] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:07.910117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcb79b-1265-4aeb-8e6a-4aae0b28ebac None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:07.910515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcb79b-1265-4aeb-8e6a-4aae0b28ebac None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:07.910899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcb79b-1265-4aeb-8e6a-4aae0b28ebac None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:07.912328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 253/495] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:07.913990 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9b899-c6e2-4f3c-9e74-4587926fcf65 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:07.914445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9b899-c6e2-4f3c-9e74-4587926fcf65 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:07.914898 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9b899-c6e2-4f3c-9e74-4587926fcf65 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:07.921287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 254/496] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:07.921287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9b899-c6e2-4f3c-9e74-4587926fcf65 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:07.953528 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:08.055463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168f44f-785a-4eb0-bdd5-29ce3f996691 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:08.093719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9b899-c6e2-4f3c-9e74-4587926fcf65 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:08.104893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287449e-4c9f-44ce-aa18-a72e9f10108a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:08.105637 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287449e-4c9f-44ce-aa18-a72e9f10108a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:08.105912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287449e-4c9f-44ce-aa18-a72e9f10108a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:08.106186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287449e-4c9f-44ce-aa18-a72e9f10108a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:08.118403 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287449e-4c9f-44ce-aa18-a72e9f10108a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:08.119901 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287449e-4c9f-44ce-aa18-a72e9f10108a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:08.133018 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c26a05-924e-4084-9532-441ce484675f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:08.133696 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c26a05-924e-4084-9532-441ce484675f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:08.133974 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c26a05-924e-4084-9532-441ce484675f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:08.134247 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c26a05-924e-4084-9532-441ce484675f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:08.140073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-1287449e-4c9f-44ce-aa18-a72e9f10108a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:08.140073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:08.140073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:08.140073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:08.140073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:08.140073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:08.140073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:08.140073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:08.140073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:08.140073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:08.140073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:08.140073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:08.140073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:08.140073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:08.140073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:08.140073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:08.141988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:08.141988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:08.141988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:08.141988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:08.141988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:08.141988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:08.141988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:08.141988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:08.141988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:08.141988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:08.141988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:08.141988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:08.141988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:08.141988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:08.141988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:08.141988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:08.141988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:08.143842 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:08.143842 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:08.143842 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:08.145380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 243/497] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:08.146676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287449e-4c9f-44ce-aa18-a72e9f10108a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:08.146861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c26a05-924e-4084-9532-441ce484675f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:08.148211 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c26a05-924e-4084-9532-441ce484675f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:08.150591 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:08.175360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-30c26a05-924e-4084-9532-441ce484675f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:08.175360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:08.175360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:08.175360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:08.175360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:08.175360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:08.175360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:08.175360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:08.175360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:08.175360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:08.175360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:08.175360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:08.175360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:08.175360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:08.175360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:08.175360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:08.176618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:08.176618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:08.176618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:08.176618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:08.176618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:08.176618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:08.176618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:08.176618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:08.176618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:08.176618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:08.176618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:08.176618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:08.176618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:08.176618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:08.176618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:08.176618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:08.176618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:08.177798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:08.177798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:08.177798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:08.177798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 255/498] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:08.179244 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233cb82-518a-4852-b689-44bf65ee08f8 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:08.179747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233cb82-518a-4852-b689-44bf65ee08f8 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:08.180214 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233cb82-518a-4852-b689-44bf65ee08f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:08.205031 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-f233cb82-518a-4852-b689-44bf65ee08f8 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:08.207004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f233cb82-518a-4852-b689-44bf65ee08f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:08.278525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:08.295981 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287449e-4c9f-44ce-aa18-a72e9f10108a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:08.314016 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 256/499] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:08.316028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ac9de-f853-4357-85c8-a0f69705ecc3 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:08.316556 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ac9de-f853-4357-85c8-a0f69705ecc3 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:08.317029 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ac9de-f853-4357-85c8-a0f69705ecc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:08.345233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-ea1ac9de-f853-4357-85c8-a0f69705ecc3 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:08.347819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ea1ac9de-f853-4357-85c8-a0f69705ecc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:08.352607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97394896-4b62-47ec-b70e-f0ae94173c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:08.353493 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97394896-4b62-47ec-b70e-f0ae94173c21 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:08.353897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97394896-4b62-47ec-b70e-f0ae94173c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:08.354267 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97394896-4b62-47ec-b70e-f0ae94173c21 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:08.368390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97394896-4b62-47ec-b70e-f0ae94173c21 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:08.371617 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97394896-4b62-47ec-b70e-f0ae94173c21 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:08.395745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-97394896-4b62-47ec-b70e-f0ae94173c21 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 05:30:08.395745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:08.395745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:08.395745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:08.395745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:08.395745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:08.395745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:08.395745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:08.395745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:08.395745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:08.395745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:08.395745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:08.395745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:08.395745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:08.395745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:08.395745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:08.397274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:08.398688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:08.398688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:08.398688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:08.398688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:08.398688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:08.398688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:08.398688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:08.398688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:08.398688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 05:30:08.398688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:08.398688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 244/500] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:30:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:08.401097 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97394896-4b62-47ec-b70e-f0ae94173c21 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:08.450808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 257/501] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:08.452374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ac9de-f853-4357-85c8-a0f69705ecc3 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:08.561032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:08.610282 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97394896-4b62-47ec-b70e-f0ae94173c21 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:08.646099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ac9de-f853-4357-85c8-a0f69705ecc3 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:08.673912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c07ba8-7612-40ee-ab63-1d6b47054524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:08.675473 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c07ba8-7612-40ee-ab63-1d6b47054524 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:08.676215 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c07ba8-7612-40ee-ab63-1d6b47054524 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:08.676717 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c07ba8-7612-40ee-ab63-1d6b47054524 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:08.678068 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c07ba8-7612-40ee-ab63-1d6b47054524 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:08.678597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c07ba8-7612-40ee-ab63-1d6b47054524 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:08.681116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c07ba8-7612-40ee-ab63-1d6b47054524 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:08.701331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 245/502] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:08.701331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c07ba8-7612-40ee-ab63-1d6b47054524 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:08.711570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:08.721572 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d17be-6441-401e-a0b1-591e122a1f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:08.722936 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d17be-6441-401e-a0b1-591e122a1f89 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:08.723435 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d17be-6441-401e-a0b1-591e122a1f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:08.723896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d17be-6441-401e-a0b1-591e122a1f89 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:08.725049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d17be-6441-401e-a0b1-591e122a1f89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:08.725601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d17be-6441-401e-a0b1-591e122a1f89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:08.728172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d17be-6441-401e-a0b1-591e122a1f89 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:08.748968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 258/503] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:08.750242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d17be-6441-401e-a0b1-591e122a1f89 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:08.836075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c07ba8-7612-40ee-ab63-1d6b47054524 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:08.880607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d17be-6441-401e-a0b1-591e122a1f89 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:08.896842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:08.901121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee65b7-5782-46e0-a289-fcda2f180bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:08.902224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee65b7-5782-46e0-a289-fcda2f180bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:08.902588 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee65b7-5782-46e0-a289-fcda2f180bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:08.902877 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee65b7-5782-46e0-a289-fcda2f180bb2 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:08.903644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee65b7-5782-46e0-a289-fcda2f180bb2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:08.903953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee65b7-5782-46e0-a289-fcda2f180bb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:08.905459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee65b7-5782-46e0-a289-fcda2f180bb2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:08.921809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0d0c1-1642-49f4-8d91-d364cbd8317b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:08.922524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0d0c1-1642-49f4-8d91-d364cbd8317b None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:08.922860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0d0c1-1642-49f4-8d91-d364cbd8317b None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:08.923192 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0d0c1-1642-49f4-8d91-d364cbd8317b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:08.936014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0d0c1-1642-49f4-8d91-d364cbd8317b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:08.941437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-b2a0d0c1-1642-49f4-8d91-d364cbd8317b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:08.941437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:08.941437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:08.941437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:08.941437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:08.941437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:08.941437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:08.941437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:08.941437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:08.941437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:08.941437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:08.941437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:08.941437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:08.941437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:08.941437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:08.941437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:08.943067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:08.943067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:08.943067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:08.943067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:08.943067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:08.943067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:08.943067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:08.943067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:08.943067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:08.943067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:08.943067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:08.943067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:08.943067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:08.943067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:08.943067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:08.943067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:08.944610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:08.944610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:08.944610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:08.944610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:08.944610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:08.944610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:08.944610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:08.944610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:08.944610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:08.944610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:08.944610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:08.944610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:08.944610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 259/504] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:08.944610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0d0c1-1642-49f4-8d91-d364cbd8317b None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:08.954107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 246/505] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Sep 26 05:30:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:08.955916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee65b7-5782-46e0-a289-fcda2f180bb2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:09.041082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:09.090284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee65b7-5782-46e0-a289-fcda2f180bb2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:09.090615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0d0c1-1642-49f4-8d91-d364cbd8317b None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:09.136951 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c5b03-85ec-463a-ad6d-a430c23c6e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:09.137279 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d94c96-7c1c-4d62-aa7c-7196ded7e117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:09.137643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d94c96-7c1c-4d62-aa7c-7196ded7e117 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:09.137800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c5b03-85ec-463a-ad6d-a430c23c6e1c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:09.138029 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d94c96-7c1c-4d62-aa7c-7196ded7e117 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:09.138211 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c5b03-85ec-463a-ad6d-a430c23c6e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:09.138396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d94c96-7c1c-4d62-aa7c-7196ded7e117 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:09.138651 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c5b03-85ec-463a-ad6d-a430c23c6e1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:09.150548 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d94c96-7c1c-4d62-aa7c-7196ded7e117 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:09.152843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d94c96-7c1c-4d62-aa7c-7196ded7e117 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:09.155090 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c5b03-85ec-463a-ad6d-a430c23c6e1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:09.157570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c5b03-85ec-463a-ad6d-a430c23c6e1c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:09.175255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-73d94c96-7c1c-4d62-aa7c-7196ded7e117 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:30:09.175255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:09.175255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:09.175255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:09.175255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:09.175255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:09.175255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:09.175255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:09.175255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:09.175255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:09.175255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:09.175255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:09.175255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:09.175255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:09.175255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:09.175255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:09.176440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:09.178204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:09.178204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:09.178204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:09.178204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:09.178204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:09.178204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:09.178204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:09.178204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:09.178204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:30:09.178204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:09.178204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 260/506] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 26 05:30:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:09.179920 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d94c96-7c1c-4d62-aa7c-7196ded7e117 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:09.183992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-a85c5b03-85ec-463a-ad6d-a430c23c6e1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:09.183992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:09.183992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:09.183992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:09.183992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:09.183992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:09.183992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:09.183992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:09.183992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:09.183992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:09.183992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:09.183992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:09.183992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:09.183992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:09.183992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:09.183992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:09.185786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:09.185786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:09.185786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:09.185786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:09.185786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:09.185786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:09.185786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:09.185786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:09.185786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:09.185786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:09.185786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:09.185786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:09.185786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:09.185786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:09.185786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:09.185786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:09.185786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:09.188994 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:09.188994 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:09.188994 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:09.188994 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 247/507] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:09.188994 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c5b03-85ec-463a-ad6d-a430c23c6e1c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:09.243553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:09.336417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c5b03-85ec-463a-ad6d-a430c23c6e1c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:09.345863 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d94c96-7c1c-4d62-aa7c-7196ded7e117 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:09.360893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:09.387288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7f164-1634-4f4a-9058-e7be8189b2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:09.388019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7f164-1634-4f4a-9058-e7be8189b2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:09.388389 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7f164-1634-4f4a-9058-e7be8189b2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:09.388760 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7f164-1634-4f4a-9058-e7be8189b2c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:09.389611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7f164-1634-4f4a-9058-e7be8189b2c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:09.389988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7f164-1634-4f4a-9058-e7be8189b2c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:09.391600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7f164-1634-4f4a-9058-e7be8189b2c2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:09.392798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfaffc-ce2e-4acc-aaa8-bc7b1ded5327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:09.393604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfaffc-ce2e-4acc-aaa8-bc7b1ded5327 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:09.393935 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfaffc-ce2e-4acc-aaa8-bc7b1ded5327 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:09.394261 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfaffc-ce2e-4acc-aaa8-bc7b1ded5327 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:09.402041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 248/508] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:30:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:09.403721 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfaffc-ce2e-4acc-aaa8-bc7b1ded5327 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:09.404711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7f164-1634-4f4a-9058-e7be8189b2c2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:09.406220 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfaffc-ce2e-4acc-aaa8-bc7b1ded5327 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:09.427203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-2ccfaffc-ce2e-4acc-aaa8-bc7b1ded5327 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 26 05:30:09.427203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:09.427203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:09.427203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:09.427203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:09.427203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:09.427203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:09.427203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:09.427203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:09.427203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:09.427203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:09.427203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:09.427203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:09.427203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:09.427203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:09.427203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:09.429194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:09.430864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:09.430864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:09.430864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:09.430864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:09.430864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:09.430864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:09.430864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:09.430864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:09.430864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 26 05:30:09.430864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:09.430864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 261/509] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:30:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:09.436092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfaffc-ce2e-4acc-aaa8-bc7b1ded5327 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:09.546718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:09.551627 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7f164-1634-4f4a-9058-e7be8189b2c2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:09.558778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfaffc-ce2e-4acc-aaa8-bc7b1ded5327 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:09.591183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4e48c-3dcb-4a44-8cbf-de1d4f4dda0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:09.591813 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4e48c-3dcb-4a44-8cbf-de1d4f4dda0d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:09.592083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4e48c-3dcb-4a44-8cbf-de1d4f4dda0d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:09.592350 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4e48c-3dcb-4a44-8cbf-de1d4f4dda0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:09.623776 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cecdf2-0960-43c3-9b97-dffdeb5e4320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:09.624649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cecdf2-0960-43c3-9b97-dffdeb5e4320 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:09.625048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cecdf2-0960-43c3-9b97-dffdeb5e4320 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:09.625487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cecdf2-0960-43c3-9b97-dffdeb5e4320 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:09.626697 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cecdf2-0960-43c3-9b97-dffdeb5e4320 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:09.627043 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cecdf2-0960-43c3-9b97-dffdeb5e4320 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:09.629411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cecdf2-0960-43c3-9b97-dffdeb5e4320 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:09.649178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:09.675245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4e48c-3dcb-4a44-8cbf-de1d4f4dda0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:09.676762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4e48c-3dcb-4a44-8cbf-de1d4f4dda0d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:09.697798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 262/510] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 26 05:30:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:09.700054 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cecdf2-0960-43c3-9b97-dffdeb5e4320 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:09.833669 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 249/511] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:30:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:30:09.834799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4e48c-3dcb-4a44-8cbf-de1d4f4dda0d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:09.868976 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cecdf2-0960-43c3-9b97-dffdeb5e4320 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:09.916318 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052a183-2f88-43f3-8af0-38ae6de86802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:09.916999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052a183-2f88-43f3-8af0-38ae6de86802 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:09.917147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052a183-2f88-43f3-8af0-38ae6de86802 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:09.917448 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052a183-2f88-43f3-8af0-38ae6de86802 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:09.918206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052a183-2f88-43f3-8af0-38ae6de86802 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:09.918506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052a183-2f88-43f3-8af0-38ae6de86802 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:09.919950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052a183-2f88-43f3-8af0-38ae6de86802 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:09.934089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 263/512] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:09.934983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052a183-2f88-43f3-8af0-38ae6de86802 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:09.942385 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:09.953383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:09.969922 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4e48c-3dcb-4a44-8cbf-de1d4f4dda0d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:10.016975 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b4668-a459-4870-8beb-d5cb085f8e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:10.017796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b4668-a459-4870-8beb-d5cb085f8e61 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:10.018148 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b4668-a459-4870-8beb-d5cb085f8e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:10.018590 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b4668-a459-4870-8beb-d5cb085f8e61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:10.031754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b4668-a459-4870-8beb-d5cb085f8e61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:10.033390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b4668-a459-4870-8beb-d5cb085f8e61 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:10.055116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-6a1b4668-a459-4870-8beb-d5cb085f8e61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:10.055116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:10.055116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:10.055116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:10.055116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:10.055116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:10.055116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:10.055116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:10.055116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:10.055116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:10.055116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:10.055116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:10.055116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:10.055116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:10.055116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:10.055116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:10.056953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:10.056953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:10.056953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:10.056953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:10.056953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:10.056953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:10.056953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:10.056953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:10.056953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:10.056953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:10.056953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:10.056953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:10.056953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:10.056953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:10.056953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:10.056953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:10.056953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:10.058620 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:10.058620 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:10.058620 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:10.058620 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 250/513] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:10.058620 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b4668-a459-4870-8beb-d5cb085f8e61 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:10.075294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052a183-2f88-43f3-8af0-38ae6de86802 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:10.119133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04993d-57b3-44a6-9bd2-f24e8569d174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:10.119808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04993d-57b3-44a6-9bd2-f24e8569d174 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:10.120129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04993d-57b3-44a6-9bd2-f24e8569d174 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:10.120504 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04993d-57b3-44a6-9bd2-f24e8569d174 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:10.121541 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04993d-57b3-44a6-9bd2-f24e8569d174 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:10.121951 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04993d-57b3-44a6-9bd2-f24e8569d174 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:10.123482 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04993d-57b3-44a6-9bd2-f24e8569d174 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:10.149757 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 264/514] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Sep 26 05:30:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:10.150860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04993d-57b3-44a6-9bd2-f24e8569d174 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:10.189799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b4668-a459-4870-8beb-d5cb085f8e61 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:10.238794 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:10.247311 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f293cee-163a-4823-8382-dc806e1b1d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:10.248092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f293cee-163a-4823-8382-dc806e1b1d50 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:10.248454 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f293cee-163a-4823-8382-dc806e1b1d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:10.248810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f293cee-163a-4823-8382-dc806e1b1d50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:10.258846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:10.296245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04993d-57b3-44a6-9bd2-f24e8569d174 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:10.339638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f293cee-163a-4823-8382-dc806e1b1d50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:10.341021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe30941-9d7b-4513-a194-675e420ca8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:10.341491 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f293cee-163a-4823-8382-dc806e1b1d50 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:10.341911 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe30941-9d7b-4513-a194-675e420ca8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:10.342235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe30941-9d7b-4513-a194-675e420ca8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:10.342551 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe30941-9d7b-4513-a194-675e420ca8b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:10.356362 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe30941-9d7b-4513-a194-675e420ca8b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:10.358279 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe30941-9d7b-4513-a194-675e420ca8b0 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:10.377124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-ffe30941-9d7b-4513-a194-675e420ca8b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:10.377124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:10.377124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:10.377124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:10.377124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:10.377124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:10.377124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:10.377124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:10.377124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:10.377124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:10.377124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:10.377124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:10.377124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:10.377124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:10.377124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:10.377124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:10.379008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:10.379008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:10.379008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:10.379008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:10.379008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:10.379008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:10.379008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:10.379008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:10.379008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:10.379008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:10.379008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:10.379008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:10.379008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:10.379008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:10.379008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:10.379008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:10.379008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:10.381723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:10.381723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:10.381723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:10.381723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 265/515] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:10.381723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe30941-9d7b-4513-a194-675e420ca8b0 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:10.472458 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 251/516] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:30:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:30:10.473792 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f293cee-163a-4823-8382-dc806e1b1d50 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:10.510049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:10.511224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe30941-9d7b-4513-a194-675e420ca8b0 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:10.538736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:10.556461 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef81d6b-d7b9-4ff8-9d7f-57bb59dd67b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:10.557438 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef81d6b-d7b9-4ff8-9d7f-57bb59dd67b9 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:10.557885 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef81d6b-d7b9-4ff8-9d7f-57bb59dd67b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:10.558393 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef81d6b-d7b9-4ff8-9d7f-57bb59dd67b9 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:10.559469 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef81d6b-d7b9-4ff8-9d7f-57bb59dd67b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:10.561228 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef81d6b-d7b9-4ff8-9d7f-57bb59dd67b9 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:10.588012 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 266/517] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:30:10 2022] POST /identity/v3/users => generated 247 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:10.589957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef81d6b-d7b9-4ff8-9d7f-57bb59dd67b9 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:10.626599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f293cee-163a-4823-8382-dc806e1b1d50 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:10.673040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d2032-136a-4055-b52b-a66d5266c978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:10.673870 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d2032-136a-4055-b52b-a66d5266c978 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:10.674210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d2032-136a-4055-b52b-a66d5266c978 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:10.674563 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d2032-136a-4055-b52b-a66d5266c978 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:10.675463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d2032-136a-4055-b52b-a66d5266c978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:10.675831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d2032-136a-4055-b52b-a66d5266c978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:10.677651 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d2032-136a-4055-b52b-a66d5266c978 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:10.692656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 252/518] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:10.698619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d2032-136a-4055-b52b-a66d5266c978 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:10.754444 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef81d6b-d7b9-4ff8-9d7f-57bb59dd67b9 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:10.801857 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370284b9-86e7-4629-a3e1-433a9df79441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:10.802834 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370284b9-86e7-4629-a3e1-433a9df79441 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:10.803280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370284b9-86e7-4629-a3e1-433a9df79441 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:10.803724 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370284b9-86e7-4629-a3e1-433a9df79441 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:10.865808 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:10.884988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d2032-136a-4055-b52b-a66d5266c978 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:10.895838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:10.936049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370284b9-86e7-4629-a3e1-433a9df79441 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:10.937911 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370284b9-86e7-4629-a3e1-433a9df79441 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:10.949760 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d2a73-f0c8-4b78-a545-b8a21fb959f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:10.950795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d2a73-f0c8-4b78-a545-b8a21fb959f3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:10.951274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d2a73-f0c8-4b78-a545-b8a21fb959f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:10.951781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d2a73-f0c8-4b78-a545-b8a21fb959f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:10.952902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d2a73-f0c8-4b78-a545-b8a21fb959f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:10.953472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d2a73-f0c8-4b78-a545-b8a21fb959f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:10.956222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d2a73-f0c8-4b78-a545-b8a21fb959f3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:10.985375 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 253/519] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:10.986342 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d2a73-f0c8-4b78-a545-b8a21fb959f3 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:11.137676 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 267/520] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:30:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:30:11.139779 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370284b9-86e7-4629-a3e1-433a9df79441 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:11.163662 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d2a73-f0c8-4b78-a545-b8a21fb959f3 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:11.242251 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b162f-73cc-4ca4-8be9-5d59171e7e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:11.243475 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b162f-73cc-4ca4-8be9-5d59171e7e74 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:11.243959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b162f-73cc-4ca4-8be9-5d59171e7e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:11.244435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b162f-73cc-4ca4-8be9-5d59171e7e74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:11.266002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b162f-73cc-4ca4-8be9-5d59171e7e74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:11.268738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b162f-73cc-4ca4-8be9-5d59171e7e74 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:11.299841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-ff7b162f-73cc-4ca4-8be9-5d59171e7e74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:11.299841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:11.299841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:11.299841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:11.299841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:11.299841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:11.299841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:11.299841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:11.299841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:11.299841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:11.299841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:11.299841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:11.299841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:11.299841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:11.299841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:11.299841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:11.300812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:11.300812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:11.300812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:11.300812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:11.300812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:11.300812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:11.300812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:11.300812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:11.300812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:11.300812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:11.300812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:11.300812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:11.300812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:11.300812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:11.300812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:11.300812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:11.300812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:11.301686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:11.301686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:11.301686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:11.302900 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 254/521] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:11.304418 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b162f-73cc-4ca4-8be9-5d59171e7e74 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:11.309845 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:11.311677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:11.360312 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370284b9-86e7-4629-a3e1-433a9df79441 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:11.438747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084d901-c7bc-4068-9dcc-1c434cc21abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:11.440219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084d901-c7bc-4068-9dcc-1c434cc21abf None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:11.440826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084d901-c7bc-4068-9dcc-1c434cc21abf None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:11.441481 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084d901-c7bc-4068-9dcc-1c434cc21abf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:11.457735 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084d901-c7bc-4068-9dcc-1c434cc21abf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:11.460765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084d901-c7bc-4068-9dcc-1c434cc21abf None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:11.476377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-6084d901-c7bc-4068-9dcc-1c434cc21abf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 05:30:11.476377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:11.476377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:11.476377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:11.476377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:11.476377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:11.476377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:11.476377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:11.476377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:11.476377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:11.476377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:11.476377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:11.476377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:11.476377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:11.476377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:11.476377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:11.478467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:11.480497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:11.480497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:11.480497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:11.480497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:11.480497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:11.480497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:11.480497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:11.480497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:11.480497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 05:30:11.480497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:11.483603 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 268/522] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:11.483603 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084d901-c7bc-4068-9dcc-1c434cc21abf None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:11.498829 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b162f-73cc-4ca4-8be9-5d59171e7e74 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:11.560403 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852bcbc-9d5d-43cb-9e32-07c3e3258da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:11.561475 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852bcbc-9d5d-43cb-9e32-07c3e3258da0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:11.561962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852bcbc-9d5d-43cb-9e32-07c3e3258da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:11.562443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852bcbc-9d5d-43cb-9e32-07c3e3258da0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:11.574773 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852bcbc-9d5d-43cb-9e32-07c3e3258da0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:11.577625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852bcbc-9d5d-43cb-9e32-07c3e3258da0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:11.600271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-3852bcbc-9d5d-43cb-9e32-07c3e3258da0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:11.600271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:11.600271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:11.600271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:11.600271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:11.600271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:11.600271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:11.600271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:11.600271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:11.600271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:11.600271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:11.600271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:11.600271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:11.600271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:11.600271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:11.601265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:11.601265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:11.601265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:11.601265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:11.601265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:11.601265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:11.601265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:11.601265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:11.601265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:11.601265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:11.601265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:11.601265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:11.601265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:11.601265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:11.601265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:11.601265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:11.601265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:11.602112 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:11.602112 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:11.602112 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:11.602112 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:11.602112 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:11.602112 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:11.602112 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:11.602112 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:11.602531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 255/523] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:11.603724 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852bcbc-9d5d-43cb-9e32-07c3e3258da0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:11.635585 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084d901-c7bc-4068-9dcc-1c434cc21abf None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:11.697830 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d0732-714a-4c8a-80b6-57dd5751d180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:11.698824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d0732-714a-4c8a-80b6-57dd5751d180 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:11.700039 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d0732-714a-4c8a-80b6-57dd5751d180 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:11.700520 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d0732-714a-4c8a-80b6-57dd5751d180 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:11.714784 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:11.715249 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:11.723344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d0732-714a-4c8a-80b6-57dd5751d180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:11.726384 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d0732-714a-4c8a-80b6-57dd5751d180 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:11.751280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-274d0732-714a-4c8a-80b6-57dd5751d180 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:11.751280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:11.751280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:11.751280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:11.751280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:11.751280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:11.751280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:11.751280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:11.751280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:11.751280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:11.751280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:11.751280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:11.751280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:11.751280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:11.751280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:11.751280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:11.752588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:11.752588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:11.752588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:11.752588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:11.752588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:11.752588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:11.752588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:11.752588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:11.752588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:11.752588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:11.752588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:11.752588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:11.752588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:11.752588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:11.752588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:11.752588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:11.752588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:11.755424 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:11.755424 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:11.755424 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:11.755424 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 269/524] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:11.755424 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d0732-714a-4c8a-80b6-57dd5751d180 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:11.795693 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852bcbc-9d5d-43cb-9e32-07c3e3258da0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:11.867869 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1453aae-57c1-4f9d-8eed-150bee2001fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:11.869370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1453aae-57c1-4f9d-8eed-150bee2001fa None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:11.869873 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1453aae-57c1-4f9d-8eed-150bee2001fa None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:11.870422 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1453aae-57c1-4f9d-8eed-150bee2001fa None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:11.871740 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1453aae-57c1-4f9d-8eed-150bee2001fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:11.872592 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1453aae-57c1-4f9d-8eed-150bee2001fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:11.876617 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1453aae-57c1-4f9d-8eed-150bee2001fa None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:11.900535 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 256/525] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:11.902126 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1453aae-57c1-4f9d-8eed-150bee2001fa None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:11.937385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d0732-714a-4c8a-80b6-57dd5751d180 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:11.992779 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdb881-90c2-41f8-9105-22170fb61b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:11.993759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdb881-90c2-41f8-9105-22170fb61b6a None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:11.994205 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdb881-90c2-41f8-9105-22170fb61b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:11.994597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdb881-90c2-41f8-9105-22170fb61b6a None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:11.995480 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdb881-90c2-41f8-9105-22170fb61b6a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:11.996010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdb881-90c2-41f8-9105-22170fb61b6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:11.997725 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdb881-90c2-41f8-9105-22170fb61b6a None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:12.026507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 270/526] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Sep 26 05:30:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:12.026507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdb881-90c2-41f8-9105-22170fb61b6a None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:12.087368 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1453aae-57c1-4f9d-8eed-150bee2001fa None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:12.108806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:12.125061 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:12.140769 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0465fc-7b1e-4936-a86b-9a90e39e733a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:12.141732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0465fc-7b1e-4936-a86b-9a90e39e733a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:12.142142 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0465fc-7b1e-4936-a86b-9a90e39e733a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:12.142530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0465fc-7b1e-4936-a86b-9a90e39e733a None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:12.144118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0465fc-7b1e-4936-a86b-9a90e39e733a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:12.144118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0465fc-7b1e-4936-a86b-9a90e39e733a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:12.145762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0465fc-7b1e-4936-a86b-9a90e39e733a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:12.178014 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 257/527] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:12.179035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0465fc-7b1e-4936-a86b-9a90e39e733a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:12.203560 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdb881-90c2-41f8-9105-22170fb61b6a None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:12.274330 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f013dd1-731b-4454-b378-52ad4d3f4fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:12.275161 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f013dd1-731b-4454-b378-52ad4d3f4fef None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:12.275582 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f013dd1-731b-4454-b378-52ad4d3f4fef None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:12.275989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f013dd1-731b-4454-b378-52ad4d3f4fef None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:12.277022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f013dd1-731b-4454-b378-52ad4d3f4fef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:12.277516 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f013dd1-731b-4454-b378-52ad4d3f4fef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:12.279729 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f013dd1-731b-4454-b378-52ad4d3f4fef None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:12.305635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 271/528] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:12.308222 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f013dd1-731b-4454-b378-52ad4d3f4fef None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:12.340293 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0465fc-7b1e-4936-a86b-9a90e39e733a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:12.403116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e89be6-25b9-4103-aca7-04529a4c383b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:12.404220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e89be6-25b9-4103-aca7-04529a4c383b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:12.404545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e89be6-25b9-4103-aca7-04529a4c383b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:12.404861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e89be6-25b9-4103-aca7-04529a4c383b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:12.417729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e89be6-25b9-4103-aca7-04529a4c383b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:12.420407 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e89be6-25b9-4103-aca7-04529a4c383b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:12.440237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b6e89be6-25b9-4103-aca7-04529a4c383b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:12.440237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:12.440237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:12.440237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:12.440237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:12.440237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:12.440237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:12.440237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:12.440237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:12.440237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:12.440237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:12.440237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:12.440237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:12.440237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:12.440237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:12.442004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:12.442004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:12.442004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:12.442004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:12.442004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:12.442004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:12.442004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:12.442004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:12.442004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:12.442004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:12.442004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:12.442004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:12.442004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:12.442004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:12.442004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:12.442004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:12.442004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:12.451607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:12.451607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:12.451607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:12.451607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:12.451607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:12.451607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:12.451607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:12.451607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:12.451607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 258/529] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:12.451607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e89be6-25b9-4103-aca7-04529a4c383b None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:12.500778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f013dd1-731b-4454-b378-52ad4d3f4fef None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:12.514937 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:12.517436 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:12.562345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b4ec9-add1-4c53-b7df-18833152ec3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:12.563123 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b4ec9-add1-4c53-b7df-18833152ec3f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:12.564134 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b4ec9-add1-4c53-b7df-18833152ec3f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:12.564550 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b4ec9-add1-4c53-b7df-18833152ec3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:12.607641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e89be6-25b9-4103-aca7-04529a4c383b None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:12.648111 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc91e4-f597-4b90-b50f-04a24c4008d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:12.648966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc91e4-f597-4b90-b50f-04a24c4008d2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:12.649442 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc91e4-f597-4b90-b50f-04a24c4008d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:12.649871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc91e4-f597-4b90-b50f-04a24c4008d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:12.650934 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc91e4-f597-4b90-b50f-04a24c4008d2 None admin] RBAC: Adding query filter params (role.id=881210e567974dc0aa3043393c00a838, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=46edd6aba77c4404977b1c39e8442b2b) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:12.651458 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc91e4-f597-4b90-b50f-04a24c4008d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:12.653317 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc91e4-f597-4b90-b50f-04a24c4008d2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:12.661195 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 259/530] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:30:12 2022] GET /identity/v3/role_assignments?user.id=46edd6aba77c4404977b1c39e8442b2b&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=881210e567974dc0aa3043393c00a838 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:12.661195 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc91e4-f597-4b90-b50f-04a24c4008d2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:12.683716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b4ec9-add1-4c53-b7df-18833152ec3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:12.687432 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b4ec9-add1-4c53-b7df-18833152ec3f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:12.869847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc91e4-f597-4b90-b50f-04a24c4008d2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:12.888406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:12.905853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 272/531] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:30:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:30:12.907955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b4ec9-add1-4c53-b7df-18833152ec3f None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:12.928394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:12.948905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c2b4c-f76d-491b-88ae-d364f884436f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:12.949908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c2b4c-f76d-491b-88ae-d364f884436f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:12.950367 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c2b4c-f76d-491b-88ae-d364f884436f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:12.951027 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c2b4c-f76d-491b-88ae-d364f884436f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:12.968792 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c2b4c-f76d-491b-88ae-d364f884436f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:12.973015 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c2b4c-f76d-491b-88ae-d364f884436f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:13.002944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-350c2b4c-f76d-491b-88ae-d364f884436f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:13.002944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:13.002944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:13.002944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:13.002944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:13.002944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:13.002944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:13.002944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:13.002944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:13.002944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:13.002944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:13.002944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:13.002944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:13.002944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:13.002944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:13.004570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:13.004570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:13.004570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:13.004570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:13.004570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:13.004570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:13.004570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:13.004570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:13.004570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:13.004570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:13.004570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:13.004570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:13.004570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:13.004570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:13.004570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:13.004570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:13.004570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:13.006197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:13.006197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:13.006197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:13.006197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:13.006197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:13.006197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:13.006197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:13.006197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:13.006197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 260/532] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:13.007111 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c2b4c-f76d-491b-88ae-d364f884436f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:13.137318 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b4ec9-add1-4c53-b7df-18833152ec3f None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:13.223751 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c54670-ede7-4818-8729-55ef29236eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:13.224791 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c54670-ede7-4818-8729-55ef29236eac None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:13.233927 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c54670-ede7-4818-8729-55ef29236eac None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:13.233927 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c54670-ede7-4818-8729-55ef29236eac None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:13.233927 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c54670-ede7-4818-8729-55ef29236eac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:13.233927 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c54670-ede7-4818-8729-55ef29236eac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:13.243421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c54670-ede7-4818-8729-55ef29236eac None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:13.283192 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 273/533] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:13.284905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c54670-ede7-4818-8729-55ef29236eac None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:13.320141 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c2b4c-f76d-491b-88ae-d364f884436f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:13.394670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54496149-14ac-46e5-a197-b91d5f372b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:13.395661 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54496149-14ac-46e5-a197-b91d5f372b16 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:13.396149 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54496149-14ac-46e5-a197-b91d5f372b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:13.396629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54496149-14ac-46e5-a197-b91d5f372b16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:13.402678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:13.425998 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54496149-14ac-46e5-a197-b91d5f372b16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:13.433581 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54496149-14ac-46e5-a197-b91d5f372b16 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:13.457893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-54496149-14ac-46e5-a197-b91d5f372b16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:13.457893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:13.457893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:13.457893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:13.457893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:13.457893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:13.457893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:13.457893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:13.457893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:13.457893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:13.457893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:13.457893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:13.457893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:13.457893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:13.457893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:13.457893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:13.459718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:13.459718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:13.459718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:13.459718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:13.459718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:13.459718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:13.459718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:13.459718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:13.459718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:13.459718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:13.459718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:13.459718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:13.459718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:13.459718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:13.459718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:13.459718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:13.459718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:13.463208 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:13.463208 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:13.463208 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:13.465676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 261/534] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 459 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:13.467231 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092bfcd5-1b1b-4213-992b-01b65e9eddbc None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:13.467649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092bfcd5-1b1b-4213-992b-01b65e9eddbc None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:13.468023 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092bfcd5-1b1b-4213-992b-01b65e9eddbc None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:13.473197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 262/535] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:13.473197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092bfcd5-1b1b-4213-992b-01b65e9eddbc None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:13.488405 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c54670-ede7-4818-8729-55ef29236eac None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:13.504622 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:13.554399 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15776b-96bc-4d69-ba12-3703c3267cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:13.555355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15776b-96bc-4d69-ba12-3703c3267cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:13.555822 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15776b-96bc-4d69-ba12-3703c3267cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:13.556273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15776b-96bc-4d69-ba12-3703c3267cd5 None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:13.557517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15776b-96bc-4d69-ba12-3703c3267cd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:13.565396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15776b-96bc-4d69-ba12-3703c3267cd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:13.565396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15776b-96bc-4d69-ba12-3703c3267cd5 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:13.601013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 274/536] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:13.603275 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15776b-96bc-4d69-ba12-3703c3267cd5 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:13.711912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092bfcd5-1b1b-4213-992b-01b65e9eddbc None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:13.795321 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15776b-96bc-4d69-ba12-3703c3267cd5 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:13.797555 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b7cf7-b241-49cd-81f8-20279aa86cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:13.798989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b7cf7-b241-49cd-81f8-20279aa86cdf None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:13.799531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b7cf7-b241-49cd-81f8-20279aa86cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:13.800059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b7cf7-b241-49cd-81f8-20279aa86cdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:13.801390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631b7cf7-b241-49cd-81f8-20279aa86cdf None admin] RBAC: Adding query filter params (role.id=881210e567974dc0aa3043393c00a838, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=92556048cbd04c409c3f4b6fd645627f) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:13.801872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631b7cf7-b241-49cd-81f8-20279aa86cdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:13.804848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631b7cf7-b241-49cd-81f8-20279aa86cdf None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:13.815568 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 263/537] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:30:13 2022] GET /identity/v3/role_assignments?user.id=92556048cbd04c409c3f4b6fd645627f&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=881210e567974dc0aa3043393c00a838 => generated 259 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:13.817780 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007eb99-7d87-4d90-9364-9875e30cd14c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:13.818240 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007eb99-7d87-4d90-9364-9875e30cd14c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:13.818669 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007eb99-7d87-4d90-9364-9875e30cd14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:13.858472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:13.861385 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-5007eb99-7d87-4d90-9364-9875e30cd14c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:13.863997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5007eb99-7d87-4d90-9364-9875e30cd14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:13.874709 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9331bea-382e-47ad-946e-cb242f86e96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:13.875995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9331bea-382e-47ad-946e-cb242f86e96e None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:13.876682 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9331bea-382e-47ad-946e-cb242f86e96e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:13.877381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9331bea-382e-47ad-946e-cb242f86e96e None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:13.878697 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9331bea-382e-47ad-946e-cb242f86e96e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:13.879420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9331bea-382e-47ad-946e-cb242f86e96e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:13.882306 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9331bea-382e-47ad-946e-cb242f86e96e None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:13.929178 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 275/538] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:13.929178 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9331bea-382e-47ad-946e-cb242f86e96e None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:14.110662 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 264/539] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:14.115645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007eb99-7d87-4d90-9364-9875e30cd14c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:14.122490 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9331bea-382e-47ad-946e-cb242f86e96e None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:14.193173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d69dc-5c9a-4edd-b4d2-f588865283a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:14.193173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d69dc-5c9a-4edd-b4d2-f588865283a7 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:14.193173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d69dc-5c9a-4edd-b4d2-f588865283a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:14.193173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d69dc-5c9a-4edd-b4d2-f588865283a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:14.194035 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d69dc-5c9a-4edd-b4d2-f588865283a7 None admin] RBAC: Adding query filter params (role.id=881210e567974dc0aa3043393c00a838, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=90c0fabecff2457e931c8c1c36681197) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:14.194035 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d69dc-5c9a-4edd-b4d2-f588865283a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:14.197663 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:14.198295 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d69dc-5c9a-4edd-b4d2-f588865283a7 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:14.213810 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 276/540] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:30:13 2022] GET /identity/v3/role_assignments?user.id=90c0fabecff2457e931c8c1c36681197&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=881210e567974dc0aa3043393c00a838 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:14.318706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007eb99-7d87-4d90-9364-9875e30cd14c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:14.361381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064de14c-cb57-4294-bf4f-0517012bb9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:14.362782 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064de14c-cb57-4294-bf4f-0517012bb9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:14.363207 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064de14c-cb57-4294-bf4f-0517012bb9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:14.363628 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064de14c-cb57-4294-bf4f-0517012bb9c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:14.372526 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064de14c-cb57-4294-bf4f-0517012bb9c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:14.374354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064de14c-cb57-4294-bf4f-0517012bb9c0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:14.387545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-064de14c-cb57-4294-bf4f-0517012bb9c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:14.387545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:14.387545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:14.387545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:14.387545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:14.387545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:14.387545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:14.387545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:14.387545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:14.387545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:14.387545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:14.387545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:14.387545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:14.387545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:14.387545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:14.388575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:14.388575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:14.388575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:14.388575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:14.388575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:14.388575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:14.388575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:14.388575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:14.388575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:14.388575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:14.388575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:14.388575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:14.388575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:14.388575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:14.388575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:14.388575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:14.388575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:14.389561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:14.389561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:14.389561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:14.389561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:14.389561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:14.389561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:14.389561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:14.389561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:14.391714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 265/541] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:14.397916 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d69dc-5c9a-4edd-b4d2-f588865283a7 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:14.481789 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:14.537882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d69dc-5c9a-4edd-b4d2-f588865283a7 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:14.638109 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23fe96-1abf-4da7-ad04-68f60b7d71fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:14.639152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23fe96-1abf-4da7-ad04-68f60b7d71fd None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:14.639653 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23fe96-1abf-4da7-ad04-68f60b7d71fd None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:14.640155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23fe96-1abf-4da7-ad04-68f60b7d71fd None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:14.641093 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e23fe96-1abf-4da7-ad04-68f60b7d71fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:14.641719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e23fe96-1abf-4da7-ad04-68f60b7d71fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:14.643463 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e23fe96-1abf-4da7-ad04-68f60b7d71fd None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:14.669088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 277/542] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:14.682935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064de14c-cb57-4294-bf4f-0517012bb9c0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:14.745436 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23fe96-1abf-4da7-ad04-68f60b7d71fd None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:14.851681 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:14.894344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064de14c-cb57-4294-bf4f-0517012bb9c0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:14.979315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec535cd3-bcde-4a63-9f97-224a5a6268ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:14.979925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec535cd3-bcde-4a63-9f97-224a5a6268ef None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:14.980248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec535cd3-bcde-4a63-9f97-224a5a6268ef None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:14.980555 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec535cd3-bcde-4a63-9f97-224a5a6268ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:14.985188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec535cd3-bcde-4a63-9f97-224a5a6268ef None admin] RBAC: Adding query filter params (role.id=881210e567974dc0aa3043393c00a838, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=6bf0f594ce7646b7b6ddb8a8553509d7) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:14.985188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec535cd3-bcde-4a63-9f97-224a5a6268ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:14.985188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec535cd3-bcde-4a63-9f97-224a5a6268ef None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:14.992200 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 266/543] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:30:14 2022] GET /identity/v3/role_assignments?user.id=6bf0f594ce7646b7b6ddb8a8553509d7&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=881210e567974dc0aa3043393c00a838 => generated 259 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:15.026698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23fe96-1abf-4da7-ad04-68f60b7d71fd None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:15.065748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:15.103607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659730f-e405-462a-a03e-c6fe358ce4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:15.104284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659730f-e405-462a-a03e-c6fe358ce4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:15.106344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659730f-e405-462a-a03e-c6fe358ce4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:15.106344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659730f-e405-462a-a03e-c6fe358ce4b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:15.128005 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659730f-e405-462a-a03e-c6fe358ce4b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:15.133186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-f659730f-e405-462a-a03e-c6fe358ce4b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:15.133186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:15.133186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:15.133186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:15.133186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:15.133186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:15.133186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:15.133186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:15.133186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:15.133186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:15.133186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:15.133186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:15.133186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:15.133186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:15.133186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:15.133186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:15.134966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:15.134966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:15.134966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:15.134966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:15.134966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:15.134966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:15.134966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:15.134966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:15.134966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:15.134966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:15.134966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:15.134966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:15.134966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:15.134966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:15.134966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:15.134966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:15.136628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:15.136628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:15.136628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:15.136628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:15.136628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:15.136628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:15.136628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:15.136628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:15.136628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:15.136628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:15.136628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:15.136628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:15.142987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 278/544] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:30:15.150217 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec535cd3-bcde-4a63-9f97-224a5a6268ef None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:15.495054 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec535cd3-bcde-4a63-9f97-224a5a6268ef None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:15.594357 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5738d5-a0df-496d-b2b6-4714d7a452dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:15.595406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5738d5-a0df-496d-b2b6-4714d7a452dc None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:15.595898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5738d5-a0df-496d-b2b6-4714d7a452dc None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:15.596385 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5738d5-a0df-496d-b2b6-4714d7a452dc None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:15.597988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5738d5-a0df-496d-b2b6-4714d7a452dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:15.598528 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5738d5-a0df-496d-b2b6-4714d7a452dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:15.601032 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5738d5-a0df-496d-b2b6-4714d7a452dc None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:15.617729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:15.625724 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 267/545] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:30:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 477 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:30:15.627143 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659730f-e405-462a-a03e-c6fe358ce4b2 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:15.976428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659730f-e405-462a-a03e-c6fe358ce4b2 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:16.056609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008b78c-1623-424b-a155-04dc3afea9fa None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:16.057052 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008b78c-1623-424b-a155-04dc3afea9fa None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:16.061657 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008b78c-1623-424b-a155-04dc3afea9fa None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:16.064447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 268/546] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:16.075615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dc9af-c6b1-4d88-b328-644234348e6e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:16.075863 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb10714-c7b7-4b40-91d5-f43f6423f22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:16.076238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dc9af-c6b1-4d88-b328-644234348e6e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:16.076525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dc9af-c6b1-4d88-b328-644234348e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:16.076820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb10714-c7b7-4b40-91d5-f43f6423f22b None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:16.077285 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb10714-c7b7-4b40-91d5-f43f6423f22b None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:16.077714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb10714-c7b7-4b40-91d5-f43f6423f22b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:16.110027 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb10714-c7b7-4b40-91d5-f43f6423f22b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:16.114223 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb10714-c7b7-4b40-91d5-f43f6423f22b None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:16.139235 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c85dc9af-c6b1-4d88-b328-644234348e6e None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:16.141466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c85dc9af-c6b1-4d88-b328-644234348e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:16.169073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-0fb10714-c7b7-4b40-91d5-f43f6423f22b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:16.169073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:16.169073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:16.169073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:16.169073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:16.169073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:16.169073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:16.169073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:16.169073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:16.169073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:16.169073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:16.169073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:16.169073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:16.169073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:16.169073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:16.169073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:16.170992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:16.170992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:16.170992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:16.170992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:16.170992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:16.170992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:16.170992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:16.170992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:16.170992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:16.170992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:16.170992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:16.170992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:16.170992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:16.170992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:16.170992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:16.170992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:16.170992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:16.172692 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:16.172692 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:16.172692 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:16.177583 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 279/547] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 551 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:16.184387 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb10714-c7b7-4b40-91d5-f43f6423f22b None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:16.283867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:16.327287 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 269/548] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:16.414940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb10714-c7b7-4b40-91d5-f43f6423f22b None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:16.468421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262509f-79a9-4f5c-832b-647749e068be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:16.469045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262509f-79a9-4f5c-832b-647749e068be None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:16.469350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262509f-79a9-4f5c-832b-647749e068be None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:16.469619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262509f-79a9-4f5c-832b-647749e068be None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:16.470273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262509f-79a9-4f5c-832b-647749e068be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:16.470569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262509f-79a9-4f5c-832b-647749e068be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:16.472307 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262509f-79a9-4f5c-832b-647749e068be None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:16.491902 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 280/549] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:16.502616 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dc9af-c6b1-4d88-b328-644234348e6e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:16.792642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dc9af-c6b1-4d88-b328-644234348e6e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:16.887088 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:16.904416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff7e94-437a-49ab-8b5d-454c06484efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:16.905725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff7e94-437a-49ab-8b5d-454c06484efc None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:16.906517 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff7e94-437a-49ab-8b5d-454c06484efc None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:16.907012 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff7e94-437a-49ab-8b5d-454c06484efc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:16.928025 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff7e94-437a-49ab-8b5d-454c06484efc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:16.930666 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff7e94-437a-49ab-8b5d-454c06484efc None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:16.973160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-0dff7e94-437a-49ab-8b5d-454c06484efc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:30:16.973160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:16.973160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:16.973160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:16.973160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:16.973160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:16.973160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:16.973160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:16.973160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:16.973160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:16.973160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:16.973160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:16.973160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:16.973160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:16.973160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:16.973160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:16.974159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:16.975095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:16.975095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:16.975095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:16.975095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:16.975095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:16.975095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:16.975095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:16.975095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:16.975095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:30:16.975095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:16.976109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 270/550] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:30:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 474 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:30:16.984792 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262509f-79a9-4f5c-832b-647749e068be None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:17.009803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff7e94-437a-49ab-8b5d-454c06484efc None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:17.251025 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff7e94-437a-49ab-8b5d-454c06484efc None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:17.265514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:17.322987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262509f-79a9-4f5c-832b-647749e068be None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:17.342561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cf6f6-7eae-41d7-81f8-ea48121680b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:17.344140 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cf6f6-7eae-41d7-81f8-ea48121680b1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:17.344825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cf6f6-7eae-41d7-81f8-ea48121680b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:17.345563 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cf6f6-7eae-41d7-81f8-ea48121680b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:17.386082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cf6f6-7eae-41d7-81f8-ea48121680b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:17.411766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-685cf6f6-7eae-41d7-81f8-ea48121680b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:17.411766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:17.411766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:17.411766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:17.411766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:17.411766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:17.411766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:17.411766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:17.411766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:17.411766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:17.411766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:17.411766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:17.411766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:17.411766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:17.411766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:17.411766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:17.413844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:17.413844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:17.413844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:17.413844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:17.413844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:17.413844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:17.413844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:17.413844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:17.413844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:17.413844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:17.413844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:17.413844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:17.413844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:17.413844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:17.413844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:17.413844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:17.415400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:17.415400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:17.415400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:17.415400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:17.415400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:17.415400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:17.415400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:17.415400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:17.415400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:17.415400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:17.415400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:17.415400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:17.423568 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 271/551] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:17.433699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cf6f6-7eae-41d7-81f8-ea48121680b1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:17.481685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde7628-cf6b-4882-8d93-08e108ddcc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:17.483238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde7628-cf6b-4882-8d93-08e108ddcc6a None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:17.484007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde7628-cf6b-4882-8d93-08e108ddcc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:17.484783 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde7628-cf6b-4882-8d93-08e108ddcc6a None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:17.486382 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde7628-cf6b-4882-8d93-08e108ddcc6a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:17.489618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde7628-cf6b-4882-8d93-08e108ddcc6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:17.494618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde7628-cf6b-4882-8d93-08e108ddcc6a None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:17.543021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 281/552] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Sep 26 05:30:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 559 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:17.551925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde7628-cf6b-4882-8d93-08e108ddcc6a None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:17.604202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:17.726376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cf6f6-7eae-41d7-81f8-ea48121680b1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:17.818887 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a29086-f6f9-4b88-9d36-5133809afe09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:17.820873 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a29086-f6f9-4b88-9d36-5133809afe09 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:17.820873 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a29086-f6f9-4b88-9d36-5133809afe09 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:17.820873 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a29086-f6f9-4b88-9d36-5133809afe09 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:17.820873 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a29086-f6f9-4b88-9d36-5133809afe09 None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:17.821534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a29086-f6f9-4b88-9d36-5133809afe09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:17.825172 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a29086-f6f9-4b88-9d36-5133809afe09 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:17.840247 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde7628-cf6b-4882-8d93-08e108ddcc6a None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:17.841435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 272/553] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:30:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:17.847396 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a29086-f6f9-4b88-9d36-5133809afe09 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:17.947770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44b56f-ec2b-42a5-967f-19bc0b875dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:17.948768 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44b56f-ec2b-42a5-967f-19bc0b875dda None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:17.949305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44b56f-ec2b-42a5-967f-19bc0b875dda None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:17.949765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44b56f-ec2b-42a5-967f-19bc0b875dda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:17.984361 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:18.074012 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a29086-f6f9-4b88-9d36-5133809afe09 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:18.081481 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a44b56f-ec2b-42a5-967f-19bc0b875dda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:18.084221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a44b56f-ec2b-42a5-967f-19bc0b875dda None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:18.087330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:18.119656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf106073-5991-4e9d-9209-5c20cbb1def8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:18.120464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf106073-5991-4e9d-9209-5c20cbb1def8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:18.121379 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf106073-5991-4e9d-9209-5c20cbb1def8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:18.121379 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf106073-5991-4e9d-9209-5c20cbb1def8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:18.140204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf106073-5991-4e9d-9209-5c20cbb1def8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:18.142350 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf106073-5991-4e9d-9209-5c20cbb1def8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:18.159401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-cf106073-5991-4e9d-9209-5c20cbb1def8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:18.159401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:18.159401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:18.159401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:18.159401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:18.159401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:18.159401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:18.159401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:18.159401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:18.159401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:18.159401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:18.159401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:18.159401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:18.159401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:18.159401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:18.159401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:18.161042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:18.161042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:18.161042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:18.161042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:18.161042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:18.161042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:18.161042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:18.161042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:18.161042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:18.161042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:18.161042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:18.161042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:18.161042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:18.161042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:18.161042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:18.161042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:18.161042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:18.162954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:18.162954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:18.162954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:18.162954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 273/554] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:30:18.162954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0ddd2-7c34-43bc-a61a-85a3bb7dbfbd None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:18.163354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0ddd2-7c34-43bc-a61a-85a3bb7dbfbd None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:18.163718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0ddd2-7c34-43bc-a61a-85a3bb7dbfbd None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:18.169254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 274/555] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:18.172295 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517454c-f869-456c-a667-fe3275f7ebed None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:18.172586 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517454c-f869-456c-a667-fe3275f7ebed None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:18.172987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517454c-f869-456c-a667-fe3275f7ebed None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:18.174881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 275/556] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:18.175695 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517454c-f869-456c-a667-fe3275f7ebed None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:18.311636 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 282/557] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:30:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 761 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:30:18.313418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb210cb-ef34-461d-8fa3-2a1ef36e1e2a None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:18.313875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb210cb-ef34-461d-8fa3-2a1ef36e1e2a None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:18.314323 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb210cb-ef34-461d-8fa3-2a1ef36e1e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:18.345109 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-1eb210cb-ef34-461d-8fa3-2a1ef36e1e2a None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:18.349242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-1eb210cb-ef34-461d-8fa3-2a1ef36e1e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:18.374289 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517454c-f869-456c-a667-fe3275f7ebed None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:18.410357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:18.422817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444491b-3940-4af7-8246-c6e750b4c8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:18.423722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444491b-3940-4af7-8246-c6e750b4c8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:18.424206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444491b-3940-4af7-8246-c6e750b4c8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:18.424689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444491b-3940-4af7-8246-c6e750b4c8e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:18.425959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444491b-3940-4af7-8246-c6e750b4c8e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:18.426582 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444491b-3940-4af7-8246-c6e750b4c8e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:18.428851 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444491b-3940-4af7-8246-c6e750b4c8e0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:18.447369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 283/558] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:18.448899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e026f-e7c6-41c6-942e-12d6cd871ea8 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:18.449242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 276/559] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:18.449334 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e026f-e7c6-41c6-942e-12d6cd871ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:18.449613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e026f-e7c6-41c6-942e-12d6cd871ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:18.450572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444491b-3940-4af7-8246-c6e750b4c8e0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:18.479951 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-089e026f-e7c6-41c6-942e-12d6cd871ea8 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:18.481630 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-089e026f-e7c6-41c6-942e-12d6cd871ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:18.513634 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:18.596611 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 284/560] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:18.597852 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd159c4-26d0-4519-b90f-42707b1306cb None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:18.598182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd159c4-26d0-4519-b90f-42707b1306cb None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:18.598501 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd159c4-26d0-4519-b90f-42707b1306cb None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:18.600168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 285/561] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:18.601022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd159c4-26d0-4519-b90f-42707b1306cb None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:18.683954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444491b-3940-4af7-8246-c6e750b4c8e0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:18.773789 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ea41f-20c2-4094-a8de-0ef12de447f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:18.774448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ea41f-20c2-4094-a8de-0ef12de447f5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:18.774726 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ea41f-20c2-4094-a8de-0ef12de447f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:18.774997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ea41f-20c2-4094-a8de-0ef12de447f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:18.805033 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ea41f-20c2-4094-a8de-0ef12de447f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:18.807651 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ea41f-20c2-4094-a8de-0ef12de447f5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:18.832147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-2e7ea41f-20c2-4094-a8de-0ef12de447f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:18.832147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:18.832147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:18.832147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:18.832147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:18.832147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:18.832147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:18.832147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:18.832147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:18.832147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:18.832147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:18.832147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:18.832147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:18.832147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:18.832147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:18.832147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:18.833839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:18.833839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:18.833839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:18.833839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:18.833839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:18.833839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:18.833839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:18.833839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:18.833839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:18.833839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:18.833839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:18.833839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:18.833839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:18.833839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:18.833839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:18.833839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:18.833839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:18.835652 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:18.835652 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:18.835652 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:18.838164 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 277/562] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:18.839937 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcfe60-6c92-4bba-8201-c95f4bff8497 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:18.840214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcfe60-6c92-4bba-8201-c95f4bff8497 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:18.840472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcfe60-6c92-4bba-8201-c95f4bff8497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:18.848072 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd159c4-26d0-4519-b90f-42707b1306cb None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:18.848072 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:18.865884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-92bcfe60-6c92-4bba-8201-c95f4bff8497 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:18.867101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-92bcfe60-6c92-4bba-8201-c95f4bff8497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:18.912903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fa3e6-ceef-4eca-965a-fcd37027d2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:18.914658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fa3e6-ceef-4eca-965a-fcd37027d2da None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:18.915367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fa3e6-ceef-4eca-965a-fcd37027d2da None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:18.916451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fa3e6-ceef-4eca-965a-fcd37027d2da None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:18.928791 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787fa3e6-ceef-4eca-965a-fcd37027d2da None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:18.930485 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787fa3e6-ceef-4eca-965a-fcd37027d2da None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:18.943400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 278/563] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:18.944315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcfe60-6c92-4bba-8201-c95f4bff8497 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:18.945705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-787fa3e6-ceef-4eca-965a-fcd37027d2da None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 05:30:18.945705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:18.945705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:18.945705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:18.945705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:18.945705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:18.945705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:18.945705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:18.945705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:18.945705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:18.945705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:18.945705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:18.945705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:18.945705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:18.945705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:18.945705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:18.947363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:18.949095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:18.949095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:18.949095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:18.949095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:18.949095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:18.949095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:18.949095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:18.949095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:18.949095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 05:30:18.949095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:18.949095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 286/564] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:30:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:18.949095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fa3e6-ceef-4eca-965a-fcd37027d2da None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:18.952561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:19.084323 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fa3e6-ceef-4eca-965a-fcd37027d2da None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:19.126115 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcfe60-6c92-4bba-8201-c95f4bff8497 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:19.148940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d69f5d-ecf7-403a-80ea-47bf5bde4ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:19.151065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d69f5d-ecf7-403a-80ea-47bf5bde4ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:19.152148 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d69f5d-ecf7-403a-80ea-47bf5bde4ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:19.152819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d69f5d-ecf7-403a-80ea-47bf5bde4ea3 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:19.154471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d69f5d-ecf7-403a-80ea-47bf5bde4ea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:19.155639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d69f5d-ecf7-403a-80ea-47bf5bde4ea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:19.159306 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d69f5d-ecf7-403a-80ea-47bf5bde4ea3 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:19.166978 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385710e-14c2-461c-8282-614ac1603a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:19.167856 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385710e-14c2-461c-8282-614ac1603a08 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:19.168227 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385710e-14c2-461c-8282-614ac1603a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:19.168779 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385710e-14c2-461c-8282-614ac1603a08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:19.181181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385710e-14c2-461c-8282-614ac1603a08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:19.188466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-d385710e-14c2-461c-8282-614ac1603a08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:19.188466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:19.188466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:19.188466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:19.188466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:19.188466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:19.188466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:19.188466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:19.188466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:19.188466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:19.188466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:19.188466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:19.188466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:19.188466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:19.188466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:19.188466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:19.190307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:19.190307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:19.190307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:19.190307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:19.190307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:19.190307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:19.190307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:19.190307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:19.190307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:19.190307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:19.190307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:19.190307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:19.190307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:19.190307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:19.190307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:19.190307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:19.191667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:19.191667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:19.191667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:19.191667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:19.191667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:19.191667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:19.191667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:19.191667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:19.191667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:19.191667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:19.191667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:19.191667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:19.192901 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 287/565] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:19.193033 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 279/566] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:19.199306 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d69f5d-ecf7-403a-80ea-47bf5bde4ea3 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:19.199568 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385710e-14c2-461c-8282-614ac1603a08 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:19.242678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:19.297066 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:19.360746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385710e-14c2-461c-8282-614ac1603a08 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:19.383908 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d69f5d-ecf7-403a-80ea-47bf5bde4ea3 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:19.419510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb235d64-b985-400c-b772-adf8a84feae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:19.419773 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477241e-6319-4419-ba64-93dcc14e8443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:19.420142 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb235d64-b985-400c-b772-adf8a84feae9 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:19.420435 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb235d64-b985-400c-b772-adf8a84feae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:19.420697 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb235d64-b985-400c-b772-adf8a84feae9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:19.420809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477241e-6319-4419-ba64-93dcc14e8443 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:19.421222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477241e-6319-4419-ba64-93dcc14e8443 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:19.421443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477241e-6319-4419-ba64-93dcc14e8443 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:19.422257 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477241e-6319-4419-ba64-93dcc14e8443 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:19.422627 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477241e-6319-4419-ba64-93dcc14e8443 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:19.424405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477241e-6319-4419-ba64-93dcc14e8443 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:19.432515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb235d64-b985-400c-b772-adf8a84feae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:19.437495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-fb235d64-b985-400c-b772-adf8a84feae9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:19.437495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:19.437495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:19.437495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:19.437495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:19.437495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:19.437495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:19.437495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:19.437495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:19.437495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:19.437495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:19.437495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:19.437495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:19.437495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:19.437495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:19.437495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:19.438668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:19.438668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:19.438668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:19.438668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:19.438668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:19.438668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:19.438668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:19.438668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:19.438668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:19.438668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:19.438668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:19.438668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:19.438668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:19.438668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:19.438668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:19.438668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:19.439618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:19.439618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:19.439618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:19.439618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:19.439618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:19.439618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:19.439618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:19.439618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:19.439618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:19.439618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:19.439618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:19.439618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:19.439618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 288/567] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:19.440399 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb235d64-b985-400c-b772-adf8a84feae9 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:19.453186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 280/568] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Sep 26 05:30:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:19.453584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477241e-6319-4419-ba64-93dcc14e8443 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:19.545715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:19.575172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb235d64-b985-400c-b772-adf8a84feae9 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:19.578952 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477241e-6319-4419-ba64-93dcc14e8443 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:19.615404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:19.623858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1d2eb-a91e-4da7-8dc7-ecf972090e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:19.624624 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1d2eb-a91e-4da7-8dc7-ecf972090e3c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:19.624894 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1d2eb-a91e-4da7-8dc7-ecf972090e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:19.625245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1d2eb-a91e-4da7-8dc7-ecf972090e3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:19.632795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827929e-e34c-419d-8a71-be125a1d595c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:19.635148 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f1d2eb-a91e-4da7-8dc7-ecf972090e3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:19.636931 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f1d2eb-a91e-4da7-8dc7-ecf972090e3c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:19.639306 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827929e-e34c-419d-8a71-be125a1d595c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:19.639698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827929e-e34c-419d-8a71-be125a1d595c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:19.639944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827929e-e34c-419d-8a71-be125a1d595c None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:19.640655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827929e-e34c-419d-8a71-be125a1d595c None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:19.640980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827929e-e34c-419d-8a71-be125a1d595c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:19.642375 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827929e-e34c-419d-8a71-be125a1d595c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:19.653884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 281/569] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:30:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:19.654745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827929e-e34c-419d-8a71-be125a1d595c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:19.655553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-e3f1d2eb-a91e-4da7-8dc7-ecf972090e3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:19.655553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:19.655553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:19.655553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:19.655553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:19.655553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:19.655553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:19.655553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:19.655553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:19.655553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:19.655553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:19.655553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:19.655553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:19.655553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:19.655553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:19.657091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:19.657091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:19.657091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:19.657091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:19.657091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:19.657091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:19.657091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:19.657091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:19.657091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:19.657091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:19.657091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:19.657091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:19.657091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:19.657091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:19.657091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:19.657091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:19.657091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:19.660385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:19.660385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:19.660385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:19.660385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:19.660385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:19.660385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:19.660385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:19.660385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:19.661850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 289/570] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:19.663049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1d2eb-a91e-4da7-8dc7-ecf972090e3c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:19.789052 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827929e-e34c-419d-8a71-be125a1d595c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:19.835991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1d2eb-a91e-4da7-8dc7-ecf972090e3c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:19.840473 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f86bc-059e-462e-a8e8-3cc471aaac45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:19.841164 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f86bc-059e-462e-a8e8-3cc471aaac45 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:19.841460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f86bc-059e-462e-a8e8-3cc471aaac45 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:19.841740 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f86bc-059e-462e-a8e8-3cc471aaac45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:19.857189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f86bc-059e-462e-a8e8-3cc471aaac45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:19.862725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-026f86bc-059e-462e-a8e8-3cc471aaac45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:19.862725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:19.862725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:19.862725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:19.862725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:19.862725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:19.862725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:19.862725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:19.862725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:19.862725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:19.862725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:19.862725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:19.862725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:19.862725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:19.862725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:19.862725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:19.864967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:19.864967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:19.864967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:19.864967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:19.864967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:19.864967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:19.864967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:19.864967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:19.864967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:19.864967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:19.864967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:19.864967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:19.864967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:19.864967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:19.864967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:19.864967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:19.866644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:19.866644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:19.866644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:19.866644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:19.866644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:19.866644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:19.866644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:19.866644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:19.866644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:19.866644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:19.866644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:19.866644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:19.866644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 282/571] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:19.866644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f86bc-059e-462e-a8e8-3cc471aaac45 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:19.895814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7276fea-6aae-4291-bbfe-b8a4a8fe2013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:19.896678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7276fea-6aae-4291-bbfe-b8a4a8fe2013 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:19.897126 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7276fea-6aae-4291-bbfe-b8a4a8fe2013 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:19.897607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7276fea-6aae-4291-bbfe-b8a4a8fe2013 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:19.898760 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7276fea-6aae-4291-bbfe-b8a4a8fe2013 None admin] RBAC: Adding query filter params (name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:19.899216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7276fea-6aae-4291-bbfe-b8a4a8fe2013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:19.900821 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7276fea-6aae-4291-bbfe-b8a4a8fe2013 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:19.909805 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 290/572] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:30:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:19.915470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7276fea-6aae-4291-bbfe-b8a4a8fe2013 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:19.923069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:19.943343 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:20.023443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f86bc-059e-462e-a8e8-3cc471aaac45 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:20.062059 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7276fea-6aae-4291-bbfe-b8a4a8fe2013 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:20.065262 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80cb0e3-6dc5-45e8-a5df-f8d837d32eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:20.065897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80cb0e3-6dc5-45e8-a5df-f8d837d32eea None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:20.066181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80cb0e3-6dc5-45e8-a5df-f8d837d32eea None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:20.066451 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80cb0e3-6dc5-45e8-a5df-f8d837d32eea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:20.078909 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80cb0e3-6dc5-45e8-a5df-f8d837d32eea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:20.080659 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80cb0e3-6dc5-45e8-a5df-f8d837d32eea None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:20.103069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-e80cb0e3-6dc5-45e8-a5df-f8d837d32eea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:20.103069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:20.103069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:20.103069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:20.103069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:20.103069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:20.103069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:20.103069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:20.103069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:20.103069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:20.103069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:20.103069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:20.103069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:20.103069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:20.103069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:20.103069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:20.105190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:20.105190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:20.105190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:20.105190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:20.105190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:20.105190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:20.105190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:20.105190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:20.105190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:20.105190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:20.105190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:20.105190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:20.105190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:20.105190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:20.105190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:20.105190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:20.105190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:20.107548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:20.107548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:20.107548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:20.107548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 283/573] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:20.107548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80cb0e3-6dc5-45e8-a5df-f8d837d32eea None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:20.122871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fea3f-5241-45b2-b0ee-9399ebcc9711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:20.123781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fea3f-5241-45b2-b0ee-9399ebcc9711 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:20.124128 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fea3f-5241-45b2-b0ee-9399ebcc9711 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:20.124359 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fea3f-5241-45b2-b0ee-9399ebcc9711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:20.141533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1fea3f-5241-45b2-b0ee-9399ebcc9711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:20.144095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1fea3f-5241-45b2-b0ee-9399ebcc9711 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:20.163101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-4c1fea3f-5241-45b2-b0ee-9399ebcc9711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:20.163101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:20.163101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:20.163101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:20.163101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:20.163101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:20.163101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:20.163101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:20.163101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:20.163101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:20.163101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:20.163101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:20.163101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:20.163101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:20.163101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:20.163101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:20.165122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:20.165122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:20.165122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:20.165122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:20.165122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:20.165122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:20.165122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:20.165122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:20.165122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:20.165122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:20.165122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:20.165122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:20.165122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:20.165122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:20.165122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:20.165122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:20.165122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:20.166685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:20.166685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:20.166685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:20.166685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 291/574] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:20.166685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fea3f-5241-45b2-b0ee-9399ebcc9711 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:20.254086 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:20.262401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80cb0e3-6dc5-45e8-a5df-f8d837d32eea None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:20.270648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:20.308891 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a1ec8-f2ab-4f7b-ac75-6f7a0b700628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:20.309903 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a1ec8-f2ab-4f7b-ac75-6f7a0b700628 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:20.310319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a1ec8-f2ab-4f7b-ac75-6f7a0b700628 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:20.311919 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a1ec8-f2ab-4f7b-ac75-6f7a0b700628 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:20.313193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a1ec8-f2ab-4f7b-ac75-6f7a0b700628 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:20.313997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a1ec8-f2ab-4f7b-ac75-6f7a0b700628 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:20.316368 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a1ec8-f2ab-4f7b-ac75-6f7a0b700628 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:20.334092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 284/575] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:20.335440 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a1ec8-f2ab-4f7b-ac75-6f7a0b700628 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:20.347274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fea3f-5241-45b2-b0ee-9399ebcc9711 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:20.393160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a2ff1-1034-4390-865b-680226264256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:20.393871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a2ff1-1034-4390-865b-680226264256 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:20.394151 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a2ff1-1034-4390-865b-680226264256 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:20.394564 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a2ff1-1034-4390-865b-680226264256 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:20.395108 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a2ff1-1034-4390-865b-680226264256 None admin] RBAC: Adding query filter params (name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:20.395445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a2ff1-1034-4390-865b-680226264256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:20.396937 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a2ff1-1034-4390-865b-680226264256 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:20.405878 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 292/576] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:30:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:20.410265 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a2ff1-1034-4390-865b-680226264256 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:20.464645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a1ec8-f2ab-4f7b-ac75-6f7a0b700628 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:20.499921 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74aea07-985a-455f-869e-576965e56cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:20.500663 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74aea07-985a-455f-869e-576965e56cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:20.501001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74aea07-985a-455f-869e-576965e56cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:20.501360 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74aea07-985a-455f-869e-576965e56cc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:20.512312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74aea07-985a-455f-869e-576965e56cc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:20.513792 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74aea07-985a-455f-869e-576965e56cc4 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:20.535359 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a2ff1-1034-4390-865b-680226264256 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:20.535861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b74aea07-985a-455f-869e-576965e56cc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:20.535861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:20.535861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:20.535861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:20.535861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:20.535861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:20.535861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:20.535861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:20.535861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:20.535861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:20.535861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:20.535861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:20.535861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:20.535861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:20.535861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:20.535861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:20.536763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:20.536763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:20.536763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:20.536763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:20.536763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:20.536763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:20.536763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:20.536763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:20.536763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:20.536763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:20.536763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:20.536763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:20.536763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:20.536763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:20.536763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:20.536763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:20.536763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:20.538130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:20.538130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:20.538130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:20.538130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 285/577] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:20.538130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74aea07-985a-455f-869e-576965e56cc4 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:20.549346 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:20.558647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:20.579975 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4320d9b-3894-459d-87d8-929d7c634576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:20.580926 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4320d9b-3894-459d-87d8-929d7c634576 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:20.581234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4320d9b-3894-459d-87d8-929d7c634576 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:20.581515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4320d9b-3894-459d-87d8-929d7c634576 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:20.582300 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4320d9b-3894-459d-87d8-929d7c634576 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:20.582760 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4320d9b-3894-459d-87d8-929d7c634576 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:20.584189 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4320d9b-3894-459d-87d8-929d7c634576 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:20.601926 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 293/578] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:20.601926 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4320d9b-3894-459d-87d8-929d7c634576 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:20.661794 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74aea07-985a-455f-869e-576965e56cc4 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:20.704102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263dd9d5-7411-4c09-b935-b6d294b2869d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:20.704719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dd9d5-7411-4c09-b935-b6d294b2869d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:20.704990 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dd9d5-7411-4c09-b935-b6d294b2869d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:20.705290 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dd9d5-7411-4c09-b935-b6d294b2869d None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:20.705937 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263dd9d5-7411-4c09-b935-b6d294b2869d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:20.706232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263dd9d5-7411-4c09-b935-b6d294b2869d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:20.707668 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263dd9d5-7411-4c09-b935-b6d294b2869d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:20.717775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4320d9b-3894-459d-87d8-929d7c634576 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:20.720535 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 286/579] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:20.721566 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263dd9d5-7411-4c09-b935-b6d294b2869d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:20.756014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee240bc-466d-48e0-b33a-4660c6d7686d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:20.756803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee240bc-466d-48e0-b33a-4660c6d7686d None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:20.757080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee240bc-466d-48e0-b33a-4660c6d7686d None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:20.757721 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee240bc-466d-48e0-b33a-4660c6d7686d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:20.758496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee240bc-466d-48e0-b33a-4660c6d7686d None admin] RBAC: Adding query filter params (role.id=881210e567974dc0aa3043393c00a838, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=00f258aba84d4dd59ebe8de973ac65c6) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:20.758793 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee240bc-466d-48e0-b33a-4660c6d7686d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:20.760245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee240bc-466d-48e0-b33a-4660c6d7686d None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:20.766389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 294/580] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:30:20 2022] GET /identity/v3/role_assignments?user.id=00f258aba84d4dd59ebe8de973ac65c6&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=881210e567974dc0aa3043393c00a838 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:20.767246 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee240bc-466d-48e0-b33a-4660c6d7686d None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:20.820366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:20.824550 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:20.866247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263dd9d5-7411-4c09-b935-b6d294b2869d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:20.914554 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee240bc-466d-48e0-b33a-4660c6d7686d None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:20.914782 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7f93b-8396-4fd3-83ba-c253d259f19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:20.915551 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7f93b-8396-4fd3-83ba-c253d259f19d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:20.915983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7f93b-8396-4fd3-83ba-c253d259f19d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:20.916494 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7f93b-8396-4fd3-83ba-c253d259f19d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:20.930123 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7f93b-8396-4fd3-83ba-c253d259f19d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:20.932001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7f93b-8396-4fd3-83ba-c253d259f19d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:20.953935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-86b7f93b-8396-4fd3-83ba-c253d259f19d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:20.953935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:20.953935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:20.953935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:20.953935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:20.953935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:20.953935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:20.953935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:20.953935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:20.953935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:20.953935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:20.953935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:20.953935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:20.953935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:20.953935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:20.953935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:20.954973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:20.954973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:20.954973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:20.954973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:20.954973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:20.954973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:20.954973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:20.954973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:20.954973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:20.954973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:20.954973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:20.954973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:20.954973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:20.954973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:20.954973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:20.954973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:20.954973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:20.955997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:20.955997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:20.955997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:20.961183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 287/581] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:20.962177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7f93b-8396-4fd3-83ba-c253d259f19d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:20.971479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55e432-6522-4994-b192-d8c6efde9c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:20.971479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55e432-6522-4994-b192-d8c6efde9c20 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:20.971479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55e432-6522-4994-b192-d8c6efde9c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:20.973243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55e432-6522-4994-b192-d8c6efde9c20 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:20.973243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55e432-6522-4994-b192-d8c6efde9c20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:20.973243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55e432-6522-4994-b192-d8c6efde9c20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:20.974804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55e432-6522-4994-b192-d8c6efde9c20 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:20.988460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 295/582] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:20.989087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55e432-6522-4994-b192-d8c6efde9c20 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:21.094883 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7f93b-8396-4fd3-83ba-c253d259f19d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:21.117735 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55e432-6522-4994-b192-d8c6efde9c20 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:21.117952 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:21.122440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:21.133446 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb36b29-7bb4-47d6-8de6-6315d14675d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:21.134097 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb36b29-7bb4-47d6-8de6-6315d14675d4 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:21.134372 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb36b29-7bb4-47d6-8de6-6315d14675d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:21.134715 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb36b29-7bb4-47d6-8de6-6315d14675d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:21.142247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb36b29-7bb4-47d6-8de6-6315d14675d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:21.143800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb36b29-7bb4-47d6-8de6-6315d14675d4 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:21.155896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-ebb36b29-7bb4-47d6-8de6-6315d14675d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:21.155896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:21.155896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:21.155896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:21.155896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:21.155896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:21.155896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:21.155896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:21.155896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:21.155896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:21.155896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:21.155896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:21.155896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:21.155896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:21.155896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:21.157020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:21.157020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:21.157020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:21.157020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:21.157020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:21.157020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:21.157020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:21.157020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:21.157020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:21.157020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:21.157020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:21.157020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:21.157020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:21.157020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:21.157020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:21.157020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:21.157020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:21.158031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:21.158031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:21.158031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:21.158031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:21.158031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:21.158031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:21.158031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:21.158031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:21.158031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 288/583] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:21.158527 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb36b29-7bb4-47d6-8de6-6315d14675d4 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:21.169455 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668d413-0f93-42f8-9cdd-e48ab4ba9edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:21.171158 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668d413-0f93-42f8-9cdd-e48ab4ba9edd None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:21.171623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668d413-0f93-42f8-9cdd-e48ab4ba9edd None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:21.172388 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668d413-0f93-42f8-9cdd-e48ab4ba9edd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:21.183475 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668d413-0f93-42f8-9cdd-e48ab4ba9edd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:21.185503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668d413-0f93-42f8-9cdd-e48ab4ba9edd None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:21.199341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-4668d413-0f93-42f8-9cdd-e48ab4ba9edd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:30:21.199341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:21.199341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:21.199341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:21.199341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:21.199341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:21.199341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:21.199341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:21.199341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:21.199341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:21.199341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:21.199341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:21.199341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:21.199341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:21.199341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:21.199341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:21.201467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:21.202468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:21.202468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:21.202468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:21.202468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:21.202468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:21.202468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:21.202468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:21.202468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:21.202468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:30:21.202468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:21.202468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 296/584] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 26 05:30:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:21.202468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668d413-0f93-42f8-9cdd-e48ab4ba9edd None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:21.297594 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb36b29-7bb4-47d6-8de6-6315d14675d4 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:21.340370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fd1bf-6097-41f9-9de0-4913d6041e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:21.340906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668d413-0f93-42f8-9cdd-e48ab4ba9edd None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:21.341098 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fd1bf-6097-41f9-9de0-4913d6041e9a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:21.341554 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fd1bf-6097-41f9-9de0-4913d6041e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:21.342057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fd1bf-6097-41f9-9de0-4913d6041e9a None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:21.343180 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fd1bf-6097-41f9-9de0-4913d6041e9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:21.343616 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fd1bf-6097-41f9-9de0-4913d6041e9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:21.345105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fd1bf-6097-41f9-9de0-4913d6041e9a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:21.363017 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 289/585] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:21.363017 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fd1bf-6097-41f9-9de0-4913d6041e9a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:21.391436 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0180e-2829-4c7d-9813-187a5451e708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:21.392100 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0180e-2829-4c7d-9813-187a5451e708 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:21.392369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0180e-2829-4c7d-9813-187a5451e708 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:21.392636 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0180e-2829-4c7d-9813-187a5451e708 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:21.404602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0180e-2829-4c7d-9813-187a5451e708 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:21.406107 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0180e-2829-4c7d-9813-187a5451e708 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:21.414450 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:21.419420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-f6f0180e-2829-4c7d-9813-187a5451e708 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 26 05:30:21.419420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:21.419420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:21.419420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:21.419420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:21.419420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:21.419420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:21.419420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:21.419420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:21.419420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:21.419420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:21.419420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:21.419420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:21.419420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:21.419420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:21.419420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:21.420956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:21.423050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:21.423050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:21.423050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:21.423050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:21.423050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:21.423050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:21.423050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:21.423050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:21.423050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 26 05:30:21.423050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:21.425467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 297/586] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:30:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:21.426614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0180e-2829-4c7d-9813-187a5451e708 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:21.461699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:21.493958 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fd1bf-6097-41f9-9de0-4913d6041e9a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:21.543290 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed87b9-699a-4f43-b224-8704136843a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:21.544274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed87b9-699a-4f43-b224-8704136843a5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:21.544722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed87b9-699a-4f43-b224-8704136843a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:21.545216 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed87b9-699a-4f43-b224-8704136843a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:21.546163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed87b9-699a-4f43-b224-8704136843a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:21.546468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed87b9-699a-4f43-b224-8704136843a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:21.548082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed87b9-699a-4f43-b224-8704136843a5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:21.559765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0180e-2829-4c7d-9813-187a5451e708 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:21.570589 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 290/587] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:21.570589 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed87b9-699a-4f43-b224-8704136843a5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:21.609628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7117ea8-1172-432a-b108-b31b9610d929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:21.610713 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7117ea8-1172-432a-b108-b31b9610d929 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:21.611234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7117ea8-1172-432a-b108-b31b9610d929 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:21.611794 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7117ea8-1172-432a-b108-b31b9610d929 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:21.613234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7117ea8-1172-432a-b108-b31b9610d929 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:21.613732 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7117ea8-1172-432a-b108-b31b9610d929 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:21.616452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7117ea8-1172-432a-b108-b31b9610d929 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:21.657259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 298/588] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 26 05:30:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:21.657259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7117ea8-1172-432a-b108-b31b9610d929 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:21.719834 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:21.724498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed87b9-699a-4f43-b224-8704136843a5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:21.766314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b4c1a-1da4-4626-ab58-676ab30d9d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:21.767167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b4c1a-1da4-4626-ab58-676ab30d9d35 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:21.767693 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b4c1a-1da4-4626-ab58-676ab30d9d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:21.768140 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b4c1a-1da4-4626-ab58-676ab30d9d35 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:21.778293 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b4c1a-1da4-4626-ab58-676ab30d9d35 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:21.780285 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b4c1a-1da4-4626-ab58-676ab30d9d35 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:21.795429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-284b4c1a-1da4-4626-ab58-676ab30d9d35 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 05:30:21.795429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:21.795429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:21.795429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:21.795429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:21.795429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:21.795429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:21.795429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:21.795429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:21.795429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:21.795429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:21.795429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:21.795429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:21.795429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:21.795429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:21.795429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:21.797544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:21.806987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:21.806987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:21.806987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:21.806987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:21.806987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:21.806987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:21.806987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:21.806987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:21.806987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 05:30:21.806987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:21.806987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 291/589] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:21.806987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b4c1a-1da4-4626-ab58-676ab30d9d35 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:21.833664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:21.869018 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7117ea8-1172-432a-b108-b31b9610d929 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:21.933964 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17e27c-7178-4e52-a9d5-2bd515b19427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:21.935058 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17e27c-7178-4e52-a9d5-2bd515b19427 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:21.935681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17e27c-7178-4e52-a9d5-2bd515b19427 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:21.935961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17e27c-7178-4e52-a9d5-2bd515b19427 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:21.937326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f17e27c-7178-4e52-a9d5-2bd515b19427 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:21.937727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f17e27c-7178-4e52-a9d5-2bd515b19427 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:21.940948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f17e27c-7178-4e52-a9d5-2bd515b19427 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:21.949395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b4c1a-1da4-4626-ab58-676ab30d9d35 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:21.977847 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 299/590] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Sep 26 05:30:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:21.979930 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17e27c-7178-4e52-a9d5-2bd515b19427 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.016455 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f5da4-5e0c-4ff3-a69d-1260cf351069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.018054 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f5da4-5e0c-4ff3-a69d-1260cf351069 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.018497 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f5da4-5e0c-4ff3-a69d-1260cf351069 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.018942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f5da4-5e0c-4ff3-a69d-1260cf351069 None admin] PATH_INFO: `/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/46edd6aba77c4404977b1c39e8442b2b/roles/881210e567974dc0aa3043393c00a838` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.050071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f5da4-5e0c-4ff3-a69d-1260cf351069 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e44d601292b47b7b66d6f3efa80b450, user_id=46edd6aba77c4404977b1c39e8442b2b, role_id=881210e567974dc0aa3043393c00a838)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.052709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f5da4-5e0c-4ff3-a69d-1260cf351069 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.073675 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 292/591] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:30:21 2022] PUT /identity/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/46edd6aba77c4404977b1c39e8442b2b/roles/881210e567974dc0aa3043393c00a838 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:22.087305 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f5da4-5e0c-4ff3-a69d-1260cf351069 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.120514 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:22.153833 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17e27c-7178-4e52-a9d5-2bd515b19427 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.203643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe0e74-126d-4125-8b39-7cdcf6226b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.205174 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe0e74-126d-4125-8b39-7cdcf6226b14 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.205909 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe0e74-126d-4125-8b39-7cdcf6226b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.206588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe0e74-126d-4125-8b39-7cdcf6226b14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.216581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:22.222818 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe0e74-126d-4125-8b39-7cdcf6226b14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.224794 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe0e74-126d-4125-8b39-7cdcf6226b14 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.244421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-7efe0e74-126d-4125-8b39-7cdcf6226b14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:22.244421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:22.244421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:22.244421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:22.244421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:22.244421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:22.244421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:22.244421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:22.244421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:22.244421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:22.244421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:22.244421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:22.244421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:22.244421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:22.244421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:22.244421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:22.245569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:22.245569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:22.245569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:22.245569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:22.245569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:22.245569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:22.245569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:22.245569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:22.245569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:22.245569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:22.245569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:22.245569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:22.245569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:22.245569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:22.245569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:22.245569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:22.245569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:22.246463 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:22.246463 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:22.246463 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:22.253644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 300/592] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:22.256603 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe0e74-126d-4125-8b39-7cdcf6226b14 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.334575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f5da4-5e0c-4ff3-a69d-1260cf351069 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.398496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe0e74-126d-4125-8b39-7cdcf6226b14 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.411724 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ed2a2-9e46-4d42-a63f-18ecd8142236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.412757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ed2a2-9e46-4d42-a63f-18ecd8142236 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.413240 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ed2a2-9e46-4d42-a63f-18ecd8142236 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.413712 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ed2a2-9e46-4d42-a63f-18ecd8142236 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.415028 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ed2a2-9e46-4d42-a63f-18ecd8142236 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:22.415532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ed2a2-9e46-4d42-a63f-18ecd8142236 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.419378 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ed2a2-9e46-4d42-a63f-18ecd8142236 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.455841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 293/593] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Sep 26 05:30:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:22.456158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ed2a2-9e46-4d42-a63f-18ecd8142236 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.475398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb8f45-ab11-42d3-80e1-e0ec118c6a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.477103 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb8f45-ab11-42d3-80e1-e0ec118c6a36 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.477103 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb8f45-ab11-42d3-80e1-e0ec118c6a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.479104 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb8f45-ab11-42d3-80e1-e0ec118c6a36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.504679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb8f45-ab11-42d3-80e1-e0ec118c6a36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.508286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb8f45-ab11-42d3-80e1-e0ec118c6a36 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.546023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-d5eb8f45-ab11-42d3-80e1-e0ec118c6a36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:22.546023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:22.546023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:22.546023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:22.546023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:22.546023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:22.546023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:22.546023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:22.546023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:22.546023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:22.546023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:22.546023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:22.546023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:22.546023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:22.546023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:22.546023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:22.547819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:22.547819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:22.547819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:22.547819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:22.547819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:22.547819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:22.547819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:22.547819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:22.547819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:22.547819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:22.547819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:22.547819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:22.547819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:22.547819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:22.547819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:22.547819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:22.547819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:22.549552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:22.549552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:22.549552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:22.549552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 301/594] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:22.549986 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb8f45-ab11-42d3-80e1-e0ec118c6a36 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.618716 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:22.647635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:22.692489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ed2a2-9e46-4d42-a63f-18ecd8142236 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.768783 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb8f45-ab11-42d3-80e1-e0ec118c6a36 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:22.779683 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2894f3-975e-4ff0-9621-3544a6a8aaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.781108 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2894f3-975e-4ff0-9621-3544a6a8aaa9 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.781930 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2894f3-975e-4ff0-9621-3544a6a8aaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.782672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2894f3-975e-4ff0-9621-3544a6a8aaa9 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.785169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2894f3-975e-4ff0-9621-3544a6a8aaa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:22.785995 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2894f3-975e-4ff0-9621-3544a6a8aaa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.789514 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2894f3-975e-4ff0-9621-3544a6a8aaa9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.824277 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 294/595] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:22.824277 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2894f3-975e-4ff0-9621-3544a6a8aaa9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:22.846025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f277a-3c36-48c7-9e11-ea7d734641dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:22.847567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f277a-3c36-48c7-9e11-ea7d734641dd None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:22.848077 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f277a-3c36-48c7-9e11-ea7d734641dd None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:22.848565 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f277a-3c36-48c7-9e11-ea7d734641dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:22.870684 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f277a-3c36-48c7-9e11-ea7d734641dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:22.873493 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f277a-3c36-48c7-9e11-ea7d734641dd None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:22.901515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-133f277a-3c36-48c7-9e11-ea7d734641dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:22.901515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:22.901515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:22.901515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:22.901515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:22.901515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:22.901515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:22.901515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:22.901515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:22.901515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:22.901515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:22.901515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:22.901515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:22.901515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:22.901515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:22.901515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:22.903274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:22.903274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:22.903274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:22.903274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:22.903274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:22.903274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:22.903274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:22.903274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:22.903274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:22.903274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:22.903274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:22.903274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:22.903274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:22.903274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:22.903274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:22.903274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:22.903274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:22.904997 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:22.904997 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:22.904997 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:22.909500 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 302/596] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:22.911105 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f277a-3c36-48c7-9e11-ea7d734641dd None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:23.040598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2894f3-975e-4ff0-9621-3544a6a8aaa9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:23.093233 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:23.104095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:23.111400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0b438-fb6b-4cd3-8bb9-8f3c635fa472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:23.112835 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0b438-fb6b-4cd3-8bb9-8f3c635fa472 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.113698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0b438-fb6b-4cd3-8bb9-8f3c635fa472 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.114419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0b438-fb6b-4cd3-8bb9-8f3c635fa472 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.115701 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f277a-3c36-48c7-9e11-ea7d734641dd None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:23.116880 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0b438-fb6b-4cd3-8bb9-8f3c635fa472 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:23.117222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0b438-fb6b-4cd3-8bb9-8f3c635fa472 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:23.120049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0b438-fb6b-4cd3-8bb9-8f3c635fa472 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:23.146177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 295/597] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:23.148235 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0b438-fb6b-4cd3-8bb9-8f3c635fa472 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:23.190313 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f320016-fea8-44f2-8fc1-00ffc569a60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:23.192745 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f320016-fea8-44f2-8fc1-00ffc569a60e None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.193751 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f320016-fea8-44f2-8fc1-00ffc569a60e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.194556 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f320016-fea8-44f2-8fc1-00ffc569a60e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.211740 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f320016-fea8-44f2-8fc1-00ffc569a60e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:23.215330 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f320016-fea8-44f2-8fc1-00ffc569a60e None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:23.242665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-2f320016-fea8-44f2-8fc1-00ffc569a60e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:23.242665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:23.242665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:23.242665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:23.242665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:23.242665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:23.242665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:23.242665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:23.242665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:23.242665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:23.242665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:23.242665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:23.242665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:23.242665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:23.242665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:23.244714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:23.244714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:23.244714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:23.244714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:23.244714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:23.244714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:23.244714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:23.244714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:23.244714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:23.244714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:23.244714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:23.244714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:23.244714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:23.244714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:23.244714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:23.244714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:23.244714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:23.246812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:23.246812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:23.246812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:23.246812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:23.246812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:23.246812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:23.246812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:23.246812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:23.260486 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 303/598] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:23.260905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f320016-fea8-44f2-8fc1-00ffc569a60e None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:23.393369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0b438-fb6b-4cd3-8bb9-8f3c635fa472 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:23.459048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbdcd9-0bb4-4b26-a525-a0f29c6d05e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:23.460059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbdcd9-0bb4-4b26-a525-a0f29c6d05e9 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.460539 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbdcd9-0bb4-4b26-a525-a0f29c6d05e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.461003 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbdcd9-0bb4-4b26-a525-a0f29c6d05e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.462250 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbdcd9-0bb4-4b26-a525-a0f29c6d05e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:23.462772 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbdcd9-0bb4-4b26-a525-a0f29c6d05e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:23.465437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbdcd9-0bb4-4b26-a525-a0f29c6d05e9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:23.501602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 296/599] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:23.503193 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f320016-fea8-44f2-8fc1-00ffc569a60e None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:23.503572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbdcd9-0bb4-4b26-a525-a0f29c6d05e9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:23.584702 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1994d-64d3-4ed3-9956-edf996a15bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:23.586184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1994d-64d3-4ed3-9956-edf996a15bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.586905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1994d-64d3-4ed3-9956-edf996a15bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.587565 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1994d-64d3-4ed3-9956-edf996a15bf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.602795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1994d-64d3-4ed3-9956-edf996a15bf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:23.606868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1994d-64d3-4ed3-9956-edf996a15bf4 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:23.612523 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:23.629680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-4bc1994d-64d3-4ed3-9956-edf996a15bf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:23.629680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:23.629680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:23.629680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:23.629680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:23.629680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:23.629680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:23.629680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:23.629680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:23.629680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:23.629680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:23.629680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:23.629680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:23.629680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:23.629680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:23.631287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:23.631287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:23.631287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:23.631287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:23.631287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:23.631287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:23.631287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:23.631287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:23.631287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:23.631287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:23.631287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:23.631287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:23.631287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:23.631287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:23.631287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:23.631287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:23.631287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:23.635067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:23.635067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:23.635067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:23.635067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:23.635067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:23.635067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:23.635067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:23.635067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:23.635067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 304/600] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:23.635067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1994d-64d3-4ed3-9956-edf996a15bf4 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:23.666435 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:23.701626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbdcd9-0bb4-4b26-a525-a0f29c6d05e9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:23.760211 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4aae3-1880-4ace-ad79-7373ff19be06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:23.761043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4aae3-1880-4ace-ad79-7373ff19be06 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.761619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4aae3-1880-4ace-ad79-7373ff19be06 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.762076 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4aae3-1880-4ace-ad79-7373ff19be06 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.763210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4aae3-1880-4ace-ad79-7373ff19be06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:23.763729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4aae3-1880-4ace-ad79-7373ff19be06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:23.766244 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4aae3-1880-4ace-ad79-7373ff19be06 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:23.785638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 297/601] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:23.787735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a08e34-25dc-4b0c-b7b9-22325914454e None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.788202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a08e34-25dc-4b0c-b7b9-22325914454e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.788655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a08e34-25dc-4b0c-b7b9-22325914454e None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.796352 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 298/602] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:23.796352 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a08e34-25dc-4b0c-b7b9-22325914454e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:23.828414 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1994d-64d3-4ed3-9956-edf996a15bf4 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:23.873191 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad7e67-91cb-4291-9bc0-a5f2b7589e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:23.874052 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad7e67-91cb-4291-9bc0-a5f2b7589e54 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:23.874824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad7e67-91cb-4291-9bc0-a5f2b7589e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:23.875229 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad7e67-91cb-4291-9bc0-a5f2b7589e54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:23.884542 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad7e67-91cb-4291-9bc0-a5f2b7589e54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:23.886695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad7e67-91cb-4291-9bc0-a5f2b7589e54 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:23.908040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-5fad7e67-91cb-4291-9bc0-a5f2b7589e54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:23.908040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:23.908040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:23.908040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:23.908040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:23.908040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:23.908040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:23.908040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:23.908040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:23.908040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:23.908040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:23.908040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:23.908040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:23.908040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:23.908040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:23.909373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:23.909373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:23.909373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:23.909373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:23.909373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:23.909373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:23.909373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:23.909373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:23.909373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:23.909373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:23.909373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:23.909373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:23.909373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:23.909373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:23.909373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:23.909373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:23.909373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:23.910643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:23.910643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:23.910643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:23.910643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:23.910643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:23.910643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:23.910643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:23.910643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:23.910643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 305/603] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:23.911567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad7e67-91cb-4291-9bc0-a5f2b7589e54 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:23.967612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a08e34-25dc-4b0c-b7b9-22325914454e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:23.985223 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:24.017971 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:24.035945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a2268-5cc8-43be-8842-08792078be5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:24.036972 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a2268-5cc8-43be-8842-08792078be5f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:24.037498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a2268-5cc8-43be-8842-08792078be5f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:24.037966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a2268-5cc8-43be-8842-08792078be5f None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:24.039207 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a2268-5cc8-43be-8842-08792078be5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:24.039755 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a2268-5cc8-43be-8842-08792078be5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:24.043222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a2268-5cc8-43be-8842-08792078be5f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:24.055894 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad7e67-91cb-4291-9bc0-a5f2b7589e54 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:24.065100 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 299/604] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:24.067671 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d5034-b1db-4d0d-94de-8276e07df658 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:24.068158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d5034-b1db-4d0d-94de-8276e07df658 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:24.068632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d5034-b1db-4d0d-94de-8276e07df658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:24.095873 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d85d5034-b1db-4d0d-94de-8276e07df658 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:24.097838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d85d5034-b1db-4d0d-94de-8276e07df658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:24.117854 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f2e17-fedf-49ed-93c9-fdc72263b35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:24.118948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f2e17-fedf-49ed-93c9-fdc72263b35c None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:24.119436 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f2e17-fedf-49ed-93c9-fdc72263b35c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:24.119920 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f2e17-fedf-49ed-93c9-fdc72263b35c None admin] PATH_INFO: `/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/92556048cbd04c409c3f4b6fd645627f/roles/881210e567974dc0aa3043393c00a838` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:24.144639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f2e17-fedf-49ed-93c9-fdc72263b35c None admin] RBAC: Authorizing `identity:create_grant(project_id=5e44d601292b47b7b66d6f3efa80b450, user_id=92556048cbd04c409c3f4b6fd645627f, role_id=881210e567974dc0aa3043393c00a838)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:24.147100 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f2e17-fedf-49ed-93c9-fdc72263b35c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:24.172271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 306/605] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:30:23 2022] PUT /identity/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/92556048cbd04c409c3f4b6fd645627f/roles/881210e567974dc0aa3043393c00a838 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:24.174188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f2e17-fedf-49ed-93c9-fdc72263b35c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:24.220486 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 300/606] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:24.225414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d5034-b1db-4d0d-94de-8276e07df658 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:24.529170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f2e17-fedf-49ed-93c9-fdc72263b35c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:24.546685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:24.554946 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d5034-b1db-4d0d-94de-8276e07df658 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:24.580251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:24.623714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7faa01-cb68-43b0-9c0f-50de0b91650b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:24.624741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7faa01-cb68-43b0-9c0f-50de0b91650b None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:24.625254 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7faa01-cb68-43b0-9c0f-50de0b91650b None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:24.625723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7faa01-cb68-43b0-9c0f-50de0b91650b None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:24.627114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7faa01-cb68-43b0-9c0f-50de0b91650b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:24.627629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7faa01-cb68-43b0-9c0f-50de0b91650b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:24.630267 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7faa01-cb68-43b0-9c0f-50de0b91650b None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:24.648901 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 307/607] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:24.659878 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7faa01-cb68-43b0-9c0f-50de0b91650b None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:24.667165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb5a8d-69d3-4c28-bd84-766417540818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:24.668363 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb5a8d-69d3-4c28-bd84-766417540818 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:24.668846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb5a8d-69d3-4c28-bd84-766417540818 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:24.673989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb5a8d-69d3-4c28-bd84-766417540818 None admin] PATH_INFO: `/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/90c0fabecff2457e931c8c1c36681197/roles/881210e567974dc0aa3043393c00a838` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:24.710085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb5a8d-69d3-4c28-bd84-766417540818 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e44d601292b47b7b66d6f3efa80b450, user_id=90c0fabecff2457e931c8c1c36681197, role_id=881210e567974dc0aa3043393c00a838)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:24.711562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb5a8d-69d3-4c28-bd84-766417540818 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:24.735265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 301/608] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:30:24 2022] PUT /identity/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/90c0fabecff2457e931c8c1c36681197/roles/881210e567974dc0aa3043393c00a838 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:24.736298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb5a8d-69d3-4c28-bd84-766417540818 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:24.883696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb5a8d-69d3-4c28-bd84-766417540818 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:24.930812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c6314-7b5f-4942-a695-d2ce8e4f1685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:24.931734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c6314-7b5f-4942-a695-d2ce8e4f1685 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:24.932135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c6314-7b5f-4942-a695-d2ce8e4f1685 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:24.932527 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c6314-7b5f-4942-a695-d2ce8e4f1685 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:24.957285 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c6314-7b5f-4942-a695-d2ce8e4f1685 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:24.965210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-6f5c6314-7b5f-4942-a695-d2ce8e4f1685 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:24.965210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:24.965210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:24.965210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:24.965210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:24.965210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:24.965210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:24.965210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:24.965210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:24.965210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:24.965210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:24.965210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:24.965210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:24.965210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:24.965210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:24.965210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:24.967325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:24.967325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:24.967325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:24.967325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:24.967325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:24.967325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:24.967325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:24.967325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:24.967325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:24.967325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:24.967325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:24.967325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:24.967325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:24.967325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:24.967325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:24.967325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:24.969219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:24.969219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:24.969219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:24.969219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:24.969219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:24.969219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:24.969219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:24.969219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:24.969219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:24.969219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:24.969219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:24.969219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:24.969219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 302/609] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:24.981172 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c6314-7b5f-4942-a695-d2ce8e4f1685 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:24.997800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:25.060659 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7faa01-cb68-43b0-9c0f-50de0b91650b None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:25.134756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c6314-7b5f-4942-a695-d2ce8e4f1685 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:25.145427 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed08a92-a94a-442b-8553-2a7161ab9f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:25.146125 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed08a92-a94a-442b-8553-2a7161ab9f05 None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:25.146398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed08a92-a94a-442b-8553-2a7161ab9f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:25.146672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed08a92-a94a-442b-8553-2a7161ab9f05 None admin] PATH_INFO: `/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/6bf0f594ce7646b7b6ddb8a8553509d7/roles/881210e567974dc0aa3043393c00a838` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:25.183093 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed08a92-a94a-442b-8553-2a7161ab9f05 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e44d601292b47b7b66d6f3efa80b450, user_id=6bf0f594ce7646b7b6ddb8a8553509d7, role_id=881210e567974dc0aa3043393c00a838)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:25.184747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed08a92-a94a-442b-8553-2a7161ab9f05 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:25.204990 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61db097-a6a5-424b-8b89-43e293e1d4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:25.206053 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61db097-a6a5-424b-8b89-43e293e1d4ed None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:25.206539 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61db097-a6a5-424b-8b89-43e293e1d4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:25.207027 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61db097-a6a5-424b-8b89-43e293e1d4ed None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:25.208297 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61db097-a6a5-424b-8b89-43e293e1d4ed None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:25.208825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61db097-a6a5-424b-8b89-43e293e1d4ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:25.211727 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61db097-a6a5-424b-8b89-43e293e1d4ed None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:25.230354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 303/610] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:30:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:30:25.239804 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61db097-a6a5-424b-8b89-43e293e1d4ed None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:25.241359 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 308/611] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:30:24 2022] PUT /identity/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/6bf0f594ce7646b7b6ddb8a8553509d7/roles/881210e567974dc0aa3043393c00a838 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:25.421824 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:25.459288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61db097-a6a5-424b-8b89-43e293e1d4ed None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:25.534029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccbb7f-0198-4482-a2ad-b070cd2199e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:25.535228 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccbb7f-0198-4482-a2ad-b070cd2199e3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:25.535711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccbb7f-0198-4482-a2ad-b070cd2199e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:25.536193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccbb7f-0198-4482-a2ad-b070cd2199e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:25.559089 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dccbb7f-0198-4482-a2ad-b070cd2199e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:25.561967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dccbb7f-0198-4482-a2ad-b070cd2199e3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:25.592331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-7dccbb7f-0198-4482-a2ad-b070cd2199e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:25.592331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:25.592331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:25.592331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:25.592331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:25.592331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:25.592331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:25.592331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:25.592331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:25.592331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:25.592331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:25.592331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:25.592331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:25.592331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:25.592331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:25.592331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:25.594109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:25.594109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:25.594109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:25.594109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:25.594109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:25.594109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:25.594109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:25.594109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:25.594109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:25.594109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:25.594109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:25.594109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:25.594109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:25.594109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:25.594109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:25.594109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:25.594109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:25.595680 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:25.595680 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:25.595680 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:25.595680 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 304/612] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:30:25.604704 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed08a92-a94a-442b-8553-2a7161ab9f05 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:25.695451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:25.839967 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed08a92-a94a-442b-8553-2a7161ab9f05 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:25.912734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265218e-25ca-480f-a5b3-ae0113cb7b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:25.913441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265218e-25ca-480f-a5b3-ae0113cb7b7b None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:25.913719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265218e-25ca-480f-a5b3-ae0113cb7b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:25.913990 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265218e-25ca-480f-a5b3-ae0113cb7b7b None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:25.914698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265218e-25ca-480f-a5b3-ae0113cb7b7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:25.915004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265218e-25ca-480f-a5b3-ae0113cb7b7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:25.916465 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265218e-25ca-480f-a5b3-ae0113cb7b7b None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:25.941307 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 309/613] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:25.946600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccbb7f-0198-4482-a2ad-b070cd2199e3 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:25.963035 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcda76-2fe7-431b-8d70-aa421248cb39 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:25.963306 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcda76-2fe7-431b-8d70-aa421248cb39 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:25.963642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcda76-2fe7-431b-8d70-aa421248cb39 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:25.965452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 310/614] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:30:25.982388 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56273b28-2ee2-4e9c-9030-01cf76509d9e None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:25.982656 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56273b28-2ee2-4e9c-9030-01cf76509d9e None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:25.982977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56273b28-2ee2-4e9c-9030-01cf76509d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:26.038412 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-56273b28-2ee2-4e9c-9030-01cf76509d9e None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:26.039676 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-56273b28-2ee2-4e9c-9030-01cf76509d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:26.182474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccbb7f-0198-4482-a2ad-b070cd2199e3 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:26.259445 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:26.283587 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9933d98-de39-4c93-8bd0-774f8e209f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:26.284471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9933d98-de39-4c93-8bd0-774f8e209f88 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:26.287074 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9933d98-de39-4c93-8bd0-774f8e209f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:26.287412 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9933d98-de39-4c93-8bd0-774f8e209f88 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:26.289681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 311/615] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:26.306585 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9933d98-de39-4c93-8bd0-774f8e209f88 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:26.308165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9933d98-de39-4c93-8bd0-774f8e209f88 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:26.333518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-c9933d98-de39-4c93-8bd0-774f8e209f88 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:30:26.333518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:26.333518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:26.333518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:26.333518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:26.333518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:26.333518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:26.333518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:26.333518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:26.333518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:26.333518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:26.333518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:26.333518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:26.333518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:26.333518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:26.333518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:26.335361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:26.337966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:26.337966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:26.337966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:26.337966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:26.337966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:26.337966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:26.337966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:26.337966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:26.337966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:30:26.337966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:26.337966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 305/616] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:30:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:26.350357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56273b28-2ee2-4e9c-9030-01cf76509d9e None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:26.413749 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:26.598698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56273b28-2ee2-4e9c-9030-01cf76509d9e None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:26.710396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba596ad-88db-4418-aa39-76918c4e4266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:26.712834 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba596ad-88db-4418-aa39-76918c4e4266 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:26.713657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba596ad-88db-4418-aa39-76918c4e4266 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:26.714469 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba596ad-88db-4418-aa39-76918c4e4266 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:26.716172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba596ad-88db-4418-aa39-76918c4e4266 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:26.717008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba596ad-88db-4418-aa39-76918c4e4266 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:26.719895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba596ad-88db-4418-aa39-76918c4e4266 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:26.772757 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 312/617] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Sep 26 05:30:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:26.786342 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9933d98-de39-4c93-8bd0-774f8e209f88 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:26.832098 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba596ad-88db-4418-aa39-76918c4e4266 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:27.011975 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9933d98-de39-4c93-8bd0-774f8e209f88 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:27.017806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:27.056049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba596ad-88db-4418-aa39-76918c4e4266 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:27.091091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e72ee-e5f7-4400-b310-8e3299fc978d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:27.091749 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e72ee-e5f7-4400-b310-8e3299fc978d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:27.092024 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e72ee-e5f7-4400-b310-8e3299fc978d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:27.092296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e72ee-e5f7-4400-b310-8e3299fc978d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:27.116558 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e72ee-e5f7-4400-b310-8e3299fc978d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:27.118081 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e72ee-e5f7-4400-b310-8e3299fc978d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:27.134044 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4050ca-e328-4039-bfe8-a4de1c1bad5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:27.134811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4050ca-e328-4039-bfe8-a4de1c1bad5b None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:27.137594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4050ca-e328-4039-bfe8-a4de1c1bad5b None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:27.137594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4050ca-e328-4039-bfe8-a4de1c1bad5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:27.157082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-120e72ee-e5f7-4400-b310-8e3299fc978d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:27.157082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:27.157082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:27.157082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:27.157082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:27.157082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:27.157082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:27.157082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:27.157082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:27.157082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:27.157082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:27.157082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:27.157082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:27.157082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:27.157082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:27.157082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:27.158676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:27.158676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:27.158676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:27.158676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:27.158676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:27.158676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:27.158676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:27.158676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:27.158676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:27.158676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:27.158676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:27.158676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:27.158676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:27.158676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:27.158676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:27.158676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:27.158676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:27.160530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:27.160530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:27.160530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:27.160530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 306/618] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:27.166619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e72ee-e5f7-4400-b310-8e3299fc978d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:27.170636 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4050ca-e328-4039-bfe8-a4de1c1bad5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:27.186678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-cb4050ca-e328-4039-bfe8-a4de1c1bad5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:27.186678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:27.186678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:27.186678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:27.186678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:27.186678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:27.186678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:27.186678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:27.186678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:27.186678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:27.186678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:27.186678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:27.186678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:27.186678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:27.186678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:27.186678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:27.188519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:27.188519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:27.188519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:27.188519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:27.188519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:27.188519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:27.188519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:27.188519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:27.188519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:27.188519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:27.188519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:27.188519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:27.188519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:27.188519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:27.188519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:27.188519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:27.190206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:27.190206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:27.190206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:27.190206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:27.190206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:27.190206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:27.190206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:27.190206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:27.190206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:27.190206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:27.190206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:27.190206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:27.194909 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 313/619] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:27.204764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4050ca-e328-4039-bfe8-a4de1c1bad5b None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:27.301333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:27.388986 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4050ca-e328-4039-bfe8-a4de1c1bad5b None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:27.510329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a570c-262f-4498-870d-ec257c067d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:27.512113 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a570c-262f-4498-870d-ec257c067d36 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:27.514709 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a570c-262f-4498-870d-ec257c067d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:27.515679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a570c-262f-4498-870d-ec257c067d36 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:27.519105 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a570c-262f-4498-870d-ec257c067d36 None admin] RBAC: Adding query filter params (name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:27.519853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a570c-262f-4498-870d-ec257c067d36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:27.522844 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a570c-262f-4498-870d-ec257c067d36 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:27.551272 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 314/620] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:30:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:30:27.555037 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e72ee-e5f7-4400-b310-8e3299fc978d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:27.559364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a570c-262f-4498-870d-ec257c067d36 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:27.637631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:27.665175 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c425b8-f6c4-4708-8f4d-33f7416cbb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:27.665983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c425b8-f6c4-4708-8f4d-33f7416cbb63 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:27.666274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c425b8-f6c4-4708-8f4d-33f7416cbb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:27.666580 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c425b8-f6c4-4708-8f4d-33f7416cbb63 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:27.667209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c425b8-f6c4-4708-8f4d-33f7416cbb63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:27.667572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c425b8-f6c4-4708-8f4d-33f7416cbb63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:27.668969 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c425b8-f6c4-4708-8f4d-33f7416cbb63 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:27.698080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 307/621] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 532 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:27.706289 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c425b8-f6c4-4708-8f4d-33f7416cbb63 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:27.811085 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a570c-262f-4498-870d-ec257c067d36 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:27.940870 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c9c0f-ac63-452b-b2ee-f5b6b14d6bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:27.944182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c9c0f-ac63-452b-b2ee-f5b6b14d6bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:27.944182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c9c0f-ac63-452b-b2ee-f5b6b14d6bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:27.944182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c9c0f-ac63-452b-b2ee-f5b6b14d6bf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:27.980985 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c9c0f-ac63-452b-b2ee-f5b6b14d6bf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:27.986117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c9c0f-ac63-452b-b2ee-f5b6b14d6bf0 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:28.013709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:28.019835 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c425b8-f6c4-4708-8f4d-33f7416cbb63 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:28.046394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-635c9c0f-ac63-452b-b2ee-f5b6b14d6bf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:28.046394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:28.046394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:28.046394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:28.046394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:28.046394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:28.046394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:28.046394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:28.046394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:28.046394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:28.046394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:28.046394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:28.046394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:28.046394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:28.046394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:28.046394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:28.048236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:28.048236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:28.048236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:28.048236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:28.048236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:28.048236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:28.048236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:28.048236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:28.048236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:28.048236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:28.048236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:28.048236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:28.048236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:28.048236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:28.048236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:28.048236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:28.048236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:28.050591 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:28.050591 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:28.050591 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:28.053883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 315/622] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 495 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:28.057180 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c9c0f-ac63-452b-b2ee-f5b6b14d6bf0 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:28.143194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276dfa6-f80a-4d3c-abea-3103ee722d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:28.145748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276dfa6-f80a-4d3c-abea-3103ee722d66 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:28.146571 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276dfa6-f80a-4d3c-abea-3103ee722d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:28.147291 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276dfa6-f80a-4d3c-abea-3103ee722d66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:28.167087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276dfa6-f80a-4d3c-abea-3103ee722d66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:28.170435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276dfa6-f80a-4d3c-abea-3103ee722d66 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:28.190298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:28.206640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-9276dfa6-f80a-4d3c-abea-3103ee722d66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:28.206640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:28.206640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:28.206640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:28.206640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:28.206640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:28.206640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:28.206640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:28.206640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:28.206640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:28.206640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:28.206640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:28.206640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:28.206640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:28.206640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:28.208373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:28.208373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:28.208373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:28.208373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:28.208373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:28.208373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:28.208373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:28.208373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:28.208373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:28.208373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:28.208373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:28.208373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:28.208373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:28.208373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:28.208373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:28.208373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:28.208373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:28.210099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:28.210099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:28.210099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:28.210099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:28.210099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:28.210099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:28.210099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:28.210099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:28.210938 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 308/623] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 505 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:28.212277 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c814ea0-8840-4145-b408-7243748994ec None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:28.212537 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c814ea0-8840-4145-b408-7243748994ec None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:28.212812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c814ea0-8840-4145-b408-7243748994ec None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:28.225364 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 309/624] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:28.225364 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c814ea0-8840-4145-b408-7243748994ec None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:28.272748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c9c0f-ac63-452b-b2ee-f5b6b14d6bf0 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:28.319079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba88f89-b815-4ad4-a9f5-735ee1d0ad55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:28.319702 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba88f89-b815-4ad4-a9f5-735ee1d0ad55 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:28.319979 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba88f89-b815-4ad4-a9f5-735ee1d0ad55 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:28.320249 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba88f89-b815-4ad4-a9f5-735ee1d0ad55 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:28.320902 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba88f89-b815-4ad4-a9f5-735ee1d0ad55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:28.321231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba88f89-b815-4ad4-a9f5-735ee1d0ad55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:28.322626 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba88f89-b815-4ad4-a9f5-735ee1d0ad55 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:28.337620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 316/625] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:28.339050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422156c0-43f2-476f-824b-b9a8a4c57b05 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:28.339409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422156c0-43f2-476f-824b-b9a8a4c57b05 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:28.339678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422156c0-43f2-476f-824b-b9a8a4c57b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:28.361181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-422156c0-43f2-476f-824b-b9a8a4c57b05 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:28.362529 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-422156c0-43f2-476f-824b-b9a8a4c57b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:28.446805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c814ea0-8840-4145-b408-7243748994ec None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:28.491817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff81274-9aef-4004-86bf-2a0c5874273a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:28.493037 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff81274-9aef-4004-86bf-2a0c5874273a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:28.494426 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff81274-9aef-4004-86bf-2a0c5874273a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:28.494426 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff81274-9aef-4004-86bf-2a0c5874273a None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:28.495704 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 317/626] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:28.495929 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422156c0-43f2-476f-824b-b9a8a4c57b05 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:28.499768 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff81274-9aef-4004-86bf-2a0c5874273a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:28.499768 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff81274-9aef-4004-86bf-2a0c5874273a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:28.499768 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff81274-9aef-4004-86bf-2a0c5874273a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:28.519768 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 310/627] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:28.521813 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d3938-702c-4324-9c8d-f7e9dc072bff None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:28.522316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d3938-702c-4324-9c8d-f7e9dc072bff None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:28.522816 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d3938-702c-4324-9c8d-f7e9dc072bff None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:28.529192 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 311/628] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:28.530444 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d3938-702c-4324-9c8d-f7e9dc072bff None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:28.549172 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:28.553744 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:28.626222 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422156c0-43f2-476f-824b-b9a8a4c57b05 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:28.668765 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d3938-702c-4324-9c8d-f7e9dc072bff None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:28.678006 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8fdd0-bcd1-477b-9118-5ba76f54772d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:28.678688 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8fdd0-bcd1-477b-9118-5ba76f54772d None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:28.679523 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8fdd0-bcd1-477b-9118-5ba76f54772d None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:28.679799 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8fdd0-bcd1-477b-9118-5ba76f54772d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:28.689073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8fdd0-bcd1-477b-9118-5ba76f54772d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:28.690763 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8fdd0-bcd1-477b-9118-5ba76f54772d None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:28.705645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-34b8fdd0-bcd1-477b-9118-5ba76f54772d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 05:30:28.705645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:28.705645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:28.705645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:28.705645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:28.705645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:28.705645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:28.705645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:28.705645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:28.705645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:28.705645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:28.705645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:28.705645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:28.705645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:28.705645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:28.705645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:28.706781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:28.707826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:28.707826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:28.707826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:28.707826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:28.707826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:28.707826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:28.707826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:28.707826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:28.707826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 05:30:28.707826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:28.711557 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 318/629] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:30:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:28.712997 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61369973-d495-4a7f-b166-56f68c3abcb5 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:28.713280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61369973-d495-4a7f-b166-56f68c3abcb5 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:28.713557 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61369973-d495-4a7f-b166-56f68c3abcb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:28.715243 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b606db-b78a-4d09-b7a7-5abb259162c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:28.716155 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b606db-b78a-4d09-b7a7-5abb259162c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:28.716997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b606db-b78a-4d09-b7a7-5abb259162c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:28.717638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b606db-b78a-4d09-b7a7-5abb259162c1 None admin] PATH_INFO: `/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/00f258aba84d4dd59ebe8de973ac65c6/roles/881210e567974dc0aa3043393c00a838` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:28.737361 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-61369973-d495-4a7f-b166-56f68c3abcb5 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:28.738613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-61369973-d495-4a7f-b166-56f68c3abcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:28.742065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b606db-b78a-4d09-b7a7-5abb259162c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e44d601292b47b7b66d6f3efa80b450, user_id=00f258aba84d4dd59ebe8de973ac65c6, role_id=881210e567974dc0aa3043393c00a838)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:28.743997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b606db-b78a-4d09-b7a7-5abb259162c1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:28.769187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 312/630] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:30:28 2022] PUT /identity/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/00f258aba84d4dd59ebe8de973ac65c6/roles/881210e567974dc0aa3043393c00a838 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:28.770851 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b606db-b78a-4d09-b7a7-5abb259162c1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:28.833781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 319/631] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:28.890499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:28.959183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b606db-b78a-4d09-b7a7-5abb259162c1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:28.985107 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61369973-d495-4a7f-b166-56f68c3abcb5 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:29.045640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee136a0-941b-41a8-9605-010e894596a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:29.046618 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee136a0-941b-41a8-9605-010e894596a0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:29.047024 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee136a0-941b-41a8-9605-010e894596a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:29.047428 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee136a0-941b-41a8-9605-010e894596a0 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:29.048648 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee136a0-941b-41a8-9605-010e894596a0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:29.049108 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee136a0-941b-41a8-9605-010e894596a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:29.051617 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee136a0-941b-41a8-9605-010e894596a0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:29.107902 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:29.118290 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 313/632] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Sep 26 05:30:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:29.122393 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee136a0-941b-41a8-9605-010e894596a0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:29.260636 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61369973-d495-4a7f-b166-56f68c3abcb5 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:29.280841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee136a0-941b-41a8-9605-010e894596a0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:29.306422 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:29.320187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44acb9-3cae-4b79-84e2-597bba2a240d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:29.320447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409b904-f34d-42d8-9bd5-c53010108c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:29.320820 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409b904-f34d-42d8-9bd5-c53010108c90 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:29.321098 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409b904-f34d-42d8-9bd5-c53010108c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:29.321392 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409b904-f34d-42d8-9bd5-c53010108c90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:29.321887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44acb9-3cae-4b79-84e2-597bba2a240d None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:29.322390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44acb9-3cae-4b79-84e2-597bba2a240d None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:29.322732 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44acb9-3cae-4b79-84e2-597bba2a240d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:29.338830 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44acb9-3cae-4b79-84e2-597bba2a240d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:29.345723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-0f44acb9-3cae-4b79-84e2-597bba2a240d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:29.345723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:29.345723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:29.345723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:29.345723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:29.345723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:29.345723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:29.345723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:29.345723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:29.345723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:29.345723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:29.345723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:29.345723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:29.345723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:29.345723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:29.345723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:29.347895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:29.347895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:29.347895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:29.347895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:29.347895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:29.347895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:29.347895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:29.347895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:29.347895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:29.347895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:29.347895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:29.347895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:29.347895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:29.347895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:29.347895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:29.347895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:29.349527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:29.349527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:29.349527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:29.349527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:29.349527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:29.349527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:29.349527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:29.349527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:29.349527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:29.349527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:29.349527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:29.349527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:29.349527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 320/633] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:29.351952 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44acb9-3cae-4b79-84e2-597bba2a240d None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:29.421366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409b904-f34d-42d8-9bd5-c53010108c90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:29.423217 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409b904-f34d-42d8-9bd5-c53010108c90 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:29.481798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:29.487662 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44acb9-3cae-4b79-84e2-597bba2a240d None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:29.533855 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67631997-0bd8-4942-874f-15217313b7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:29.534490 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67631997-0bd8-4942-874f-15217313b7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:29.534833 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67631997-0bd8-4942-874f-15217313b7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:29.535168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67631997-0bd8-4942-874f-15217313b7d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:29.541381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 314/634] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:30:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 05:30:29.541381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7cb92-daed-4bc2-a3e9-3ce34f26fe9c None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:29.541381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7cb92-daed-4bc2-a3e9-3ce34f26fe9c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:29.541381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7cb92-daed-4bc2-a3e9-3ce34f26fe9c None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:29.542580 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 315/635] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:29.543360 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7cb92-daed-4bc2-a3e9-3ce34f26fe9c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:29.550607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67631997-0bd8-4942-874f-15217313b7d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:29.556470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-67631997-0bd8-4942-874f-15217313b7d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:29.556470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:29.556470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:29.556470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:29.556470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:29.556470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:29.556470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:29.556470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:29.556470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:29.556470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:29.556470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:29.556470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:29.556470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:29.556470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:29.556470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:29.556470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:29.557517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:29.557517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:29.557517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:29.557517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:29.557517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:29.557517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:29.557517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:29.557517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:29.557517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:29.557517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:29.557517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:29.557517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:29.557517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:29.557517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:29.557517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:29.557517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:29.558418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:29.558418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:29.558418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:29.558418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:29.558418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:29.558418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:29.558418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:29.558418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:29.558418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:29.558418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:29.558418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:29.558418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:29.564664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 321/636] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:29.564664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67631997-0bd8-4942-874f-15217313b7d4 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:29.583203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:29.682351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7cb92-daed-4bc2-a3e9-3ce34f26fe9c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:29.714948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67631997-0bd8-4942-874f-15217313b7d4 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:29.722187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded799f7-bd70-4984-8303-fd972de863ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:29.722809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded799f7-bd70-4984-8303-fd972de863ec None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:29.723079 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded799f7-bd70-4984-8303-fd972de863ec None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:29.723341 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded799f7-bd70-4984-8303-fd972de863ec None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:29.724009 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded799f7-bd70-4984-8303-fd972de863ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:29.724303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded799f7-bd70-4984-8303-fd972de863ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:29.725727 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded799f7-bd70-4984-8303-fd972de863ec None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:29.741623 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 316/637] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:30:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:29.743024 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cae44d-52b5-4485-ad6c-fd940b84e9bd None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:29.743317 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cae44d-52b5-4485-ad6c-fd940b84e9bd None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:29.743576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cae44d-52b5-4485-ad6c-fd940b84e9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:29.765425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c1cae44d-52b5-4485-ad6c-fd940b84e9bd None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:29.767716 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c1cae44d-52b5-4485-ad6c-fd940b84e9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:29.769127 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cda62d-1e78-4b7f-a3c3-8df88c660fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:29.769804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cda62d-1e78-4b7f-a3c3-8df88c660fcc None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:29.770076 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cda62d-1e78-4b7f-a3c3-8df88c660fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:29.770344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cda62d-1e78-4b7f-a3c3-8df88c660fcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:29.788499 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cda62d-1e78-4b7f-a3c3-8df88c660fcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:29.791147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cda62d-1e78-4b7f-a3c3-8df88c660fcc None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:29.817806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-15cda62d-1e78-4b7f-a3c3-8df88c660fcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:29.817806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:29.817806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:29.817806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:29.817806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:29.817806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:29.817806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:29.817806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:29.817806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:29.817806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:29.817806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:29.817806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:29.817806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:29.817806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:29.817806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:29.817806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:29.818881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:29.818881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:29.818881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:29.818881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:29.818881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:29.818881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:29.818881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:29.818881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:29.818881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:29.818881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:29.818881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:29.818881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:29.818881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:29.818881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:29.818881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:29.818881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:29.818881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:29.819795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:29.819795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:29.819795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:29.819795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 322/638] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:29.821306 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cda62d-1e78-4b7f-a3c3-8df88c660fcc None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:29.840780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:29.866100 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 317/639] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:29.866100 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cae44d-52b5-4485-ad6c-fd940b84e9bd None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:29.890794 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:29.997120 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cda62d-1e78-4b7f-a3c3-8df88c660fcc None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.056637 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785b267-925a-4d09-85db-12a8b4fded76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.057644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785b267-925a-4d09-85db-12a8b4fded76 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.058114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785b267-925a-4d09-85db-12a8b4fded76 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.058395 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785b267-925a-4d09-85db-12a8b4fded76 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.059615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785b267-925a-4d09-85db-12a8b4fded76 None admin] RBAC: Adding query filter params (name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:30.060198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785b267-925a-4d09-85db-12a8b4fded76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.062755 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cae44d-52b5-4485-ad6c-fd940b84e9bd None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.066789 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785b267-925a-4d09-85db-12a8b4fded76 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.089650 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 323/640] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:30:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:30.091911 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785b267-925a-4d09-85db-12a8b4fded76 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.114353 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecb0cd-7029-4110-a31b-2746fb5036e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.115428 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecb0cd-7029-4110-a31b-2746fb5036e6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.116563 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecb0cd-7029-4110-a31b-2746fb5036e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.117066 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecb0cd-7029-4110-a31b-2746fb5036e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.130299 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecb0cd-7029-4110-a31b-2746fb5036e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.131803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecb0cd-7029-4110-a31b-2746fb5036e6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.148638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-14ecb0cd-7029-4110-a31b-2746fb5036e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:30.148638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:30.148638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:30.148638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:30.148638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:30.148638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:30.148638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:30.148638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:30.148638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:30.148638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:30.148638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:30.148638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:30.148638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:30.148638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:30.148638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:30.148638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:30.150438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:30.150438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:30.150438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:30.150438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:30.150438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:30.150438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:30.150438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:30.150438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:30.150438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:30.150438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:30.150438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:30.150438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:30.150438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:30.150438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:30.150438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:30.150438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:30.150438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:30.152897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:30.152897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:30.152897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:30.152897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 318/641] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:30.152897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecb0cd-7029-4110-a31b-2746fb5036e6 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.203556 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:30.243964 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:30.268331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785b267-925a-4d09-85db-12a8b4fded76 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.282904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecb0cd-7029-4110-a31b-2746fb5036e6 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.318282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734abd0d-ac84-4780-8f13-9036327a1625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.319007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734abd0d-ac84-4780-8f13-9036327a1625 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.319296 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734abd0d-ac84-4780-8f13-9036327a1625 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.319580 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734abd0d-ac84-4780-8f13-9036327a1625 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.320246 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734abd0d-ac84-4780-8f13-9036327a1625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:30.320524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734abd0d-ac84-4780-8f13-9036327a1625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.322003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734abd0d-ac84-4780-8f13-9036327a1625 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.325837 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b86aff-ccd8-49fc-be02-623b8c7e6030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.326487 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b86aff-ccd8-49fc-be02-623b8c7e6030 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.326761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b86aff-ccd8-49fc-be02-623b8c7e6030 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.327029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b86aff-ccd8-49fc-be02-623b8c7e6030 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.336691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 324/642] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:30.340324 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b86aff-ccd8-49fc-be02-623b8c7e6030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.341913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b86aff-ccd8-49fc-be02-623b8c7e6030 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.342055 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734abd0d-ac84-4780-8f13-9036327a1625 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.359355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-74b86aff-ccd8-49fc-be02-623b8c7e6030 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:30.359355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:30.359355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:30.359355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:30.359355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:30.359355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:30.359355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:30.359355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:30.359355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:30.359355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:30.359355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:30.359355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:30.359355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:30.359355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:30.359355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:30.359355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:30.362001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:30.362001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:30.362001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:30.362001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:30.362001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:30.362001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:30.362001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:30.362001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:30.362001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:30.362001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:30.362001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:30.362001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:30.362001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:30.362001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:30.362001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:30.362001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:30.362001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:30.363767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:30.363767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:30.363767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:30.363767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 319/643] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:30.363767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b86aff-ccd8-49fc-be02-623b8c7e6030 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.490568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734abd0d-ac84-4780-8f13-9036327a1625 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.491736 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b86aff-ccd8-49fc-be02-623b8c7e6030 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.522171 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:30.527315 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:30.535282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae1324-801e-42ed-a44a-98690b26fa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.536302 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae1324-801e-42ed-a44a-98690b26fa07 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.536928 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae1324-801e-42ed-a44a-98690b26fa07 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.537321 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae1324-801e-42ed-a44a-98690b26fa07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.547270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5115e4a-6f96-4569-a1da-c255384c2f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.548094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5115e4a-6f96-4569-a1da-c255384c2f39 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.548268 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5115e4a-6f96-4569-a1da-c255384c2f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.548421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5115e4a-6f96-4569-a1da-c255384c2f39 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.549048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5115e4a-6f96-4569-a1da-c255384c2f39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:30.549455 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5115e4a-6f96-4569-a1da-c255384c2f39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.549947 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae1324-801e-42ed-a44a-98690b26fa07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:30.550767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5115e4a-6f96-4569-a1da-c255384c2f39 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.554814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-9aae1324-801e-42ed-a44a-98690b26fa07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:30.554814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:30.554814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:30.554814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:30.554814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:30.554814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:30.554814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:30.554814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:30.554814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:30.554814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:30.554814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:30.554814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:30.554814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:30.554814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:30.554814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:30.554814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:30.561813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:30.561813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:30.561813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:30.561813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:30.561813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:30.561813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:30.561813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:30.561813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:30.561813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:30.561813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:30.561813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:30.561813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:30.561813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:30.561813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:30.561813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:30.561813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:30.563024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:30.563024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:30.563024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:30.563024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:30.563024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:30.563024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:30.563024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:30.563024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:30.563024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:30.563024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:30.563024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:30.563024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:30.563024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 325/644] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:30.563024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae1324-801e-42ed-a44a-98690b26fa07 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.570591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 320/645] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:30.571642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5115e4a-6f96-4569-a1da-c255384c2f39 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.692500 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae1324-801e-42ed-a44a-98690b26fa07 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.694222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5115e4a-6f96-4569-a1da-c255384c2f39 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.737087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98b5be-b865-4c73-971d-4b437a2dcedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.737982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98b5be-b865-4c73-971d-4b437a2dcedf None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.738152 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98b5be-b865-4c73-971d-4b437a2dcedf None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.738473 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98b5be-b865-4c73-971d-4b437a2dcedf None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.739257 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98b5be-b865-4c73-971d-4b437a2dcedf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:30.739439 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3d51e-71d7-4c45-949e-312f5913271b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.739811 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98b5be-b865-4c73-971d-4b437a2dcedf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.740078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3d51e-71d7-4c45-949e-312f5913271b None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.740426 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3d51e-71d7-4c45-949e-312f5913271b None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.740764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3d51e-71d7-4c45-949e-312f5913271b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:30.742444 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98b5be-b865-4c73-971d-4b437a2dcedf None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.749188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3d51e-71d7-4c45-949e-312f5913271b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:30.750887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3d51e-71d7-4c45-949e-312f5913271b None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:30.761114 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 321/646] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:30.763452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98b5be-b865-4c73-971d-4b437a2dcedf None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.764175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-a1d3d51e-71d7-4c45-949e-312f5913271b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:30.764175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:30.764175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:30.764175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:30.764175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:30.764175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:30.764175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:30.764175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:30.764175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:30.764175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:30.764175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:30.764175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:30.764175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:30.764175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:30.764175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:30.765900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:30.765900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:30.765900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:30.765900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:30.765900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:30.765900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:30.765900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:30.765900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:30.765900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:30.765900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:30.765900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:30.765900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:30.765900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:30.765900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:30.765900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:30.765900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:30.765900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:30.768063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:30.768063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:30.768063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:30.768063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:30.768063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:30.768063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:30.768063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:30.768063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:30.768063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 326/647] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:30.773468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3d51e-71d7-4c45-949e-312f5913271b None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:30.840753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:30.862760 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:30.928130 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3d51e-71d7-4c45-949e-312f5913271b None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.952671 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98b5be-b865-4c73-971d-4b437a2dcedf None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:30.985322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b998e-f15c-40b0-993e-9735f057cdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:30.986339 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b998e-f15c-40b0-993e-9735f057cdc5 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:30.986793 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b998e-f15c-40b0-993e-9735f057cdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:30.987322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b998e-f15c-40b0-993e-9735f057cdc5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.000251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b998e-f15c-40b0-993e-9735f057cdc5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.000863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55fa63-a6b1-4140-a8c8-66e79bae4597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.001669 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55fa63-a6b1-4140-a8c8-66e79bae4597 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.002093 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55fa63-a6b1-4140-a8c8-66e79bae4597 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.002504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55fa63-a6b1-4140-a8c8-66e79bae4597 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.002710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b998e-f15c-40b0-993e-9735f057cdc5 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.003468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55fa63-a6b1-4140-a8c8-66e79bae4597 None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:31.003902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55fa63-a6b1-4140-a8c8-66e79bae4597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.005707 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55fa63-a6b1-4140-a8c8-66e79bae4597 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.016453 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 322/648] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:30:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:31.017629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55fa63-a6b1-4140-a8c8-66e79bae4597 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.023297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-a09b998e-f15c-40b0-993e-9735f057cdc5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:30:31.023297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:31.023297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:31.023297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:31.023297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:31.023297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:31.023297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:31.023297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:31.023297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:31.023297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:31.023297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:31.023297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:31.023297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:31.023297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:31.023297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:31.023297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:31.024328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:31.030235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:31.030235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:31.030235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:31.030235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:31.030235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:31.030235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:31.030235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:31.030235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:31.030235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:30:31.030235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:31.030235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 327/649] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 26 05:30:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:31.030235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b998e-f15c-40b0-993e-9735f057cdc5 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.175026 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b998e-f15c-40b0-993e-9735f057cdc5 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.185502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:31.189943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55fa63-a6b1-4140-a8c8-66e79bae4597 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.192676 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:31.242645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d821d09-6e09-4d2c-9935-9537471202b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.243299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d821d09-6e09-4d2c-9935-9537471202b1 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.243574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d821d09-6e09-4d2c-9935-9537471202b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.243866 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d821d09-6e09-4d2c-9935-9537471202b1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.251954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b389fb-d105-4182-9cf4-50eefd1743d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.252974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b389fb-d105-4182-9cf4-50eefd1743d2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.253467 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b389fb-d105-4182-9cf4-50eefd1743d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.253630 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d821d09-6e09-4d2c-9935-9537471202b1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.253917 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b389fb-d105-4182-9cf4-50eefd1743d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.255127 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d821d09-6e09-4d2c-9935-9537471202b1 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.255327 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b389fb-d105-4182-9cf4-50eefd1743d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:31.255682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b389fb-d105-4182-9cf4-50eefd1743d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.260015 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b389fb-d105-4182-9cf4-50eefd1743d2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.269642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-7d821d09-6e09-4d2c-9935-9537471202b1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 26 05:30:31.269642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:31.269642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:31.269642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:31.269642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:31.269642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:31.269642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:31.269642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:31.269642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:31.269642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:31.269642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:31.269642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:31.269642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:31.269642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:31.269642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:31.269642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:31.271442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:31.273286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:31.273286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:31.273286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:31.273286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:31.273286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:31.273286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:31.273286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:31.273286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:31.273286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 26 05:30:31.273286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:31.275677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 328/650] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:30:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:31.276747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d821d09-6e09-4d2c-9935-9537471202b1 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.283125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 323/651] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:31.284002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b389fb-d105-4182-9cf4-50eefd1743d2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.401859 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d821d09-6e09-4d2c-9935-9537471202b1 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.437628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0a03f-9881-4a50-baca-d57135a78ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.438004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b389fb-d105-4182-9cf4-50eefd1743d2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.438179 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0a03f-9881-4a50-baca-d57135a78ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.438459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0a03f-9881-4a50-baca-d57135a78ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.438862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0a03f-9881-4a50-baca-d57135a78ad4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.451151 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0a03f-9881-4a50-baca-d57135a78ad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.452652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0a03f-9881-4a50-baca-d57135a78ad4 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.468326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-48c0a03f-9881-4a50-baca-d57135a78ad4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:31.468326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:31.468326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:31.468326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:31.468326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:31.468326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:31.468326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:31.468326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:31.468326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:31.468326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:31.468326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:31.468326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:31.468326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:31.468326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:31.468326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:31.468326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:31.469631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:31.469631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:31.469631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:31.469631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:31.469631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:31.469631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:31.469631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:31.469631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:31.469631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:31.469631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:31.469631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:31.469631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:31.469631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:31.469631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:31.469631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:31.469631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:31.469631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:31.470621 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:31.470621 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:31.470621 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:31.470621 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 329/652] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:31.470868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0a03f-9881-4a50-baca-d57135a78ad4 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.497924 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a12c8b-c4ef-4694-a915-8001aca4fdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.498879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a12c8b-c4ef-4694-a915-8001aca4fdf5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.499335 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a12c8b-c4ef-4694-a915-8001aca4fdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.499760 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a12c8b-c4ef-4694-a915-8001aca4fdf5 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.500986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a12c8b-c4ef-4694-a915-8001aca4fdf5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:31.501511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a12c8b-c4ef-4694-a915-8001aca4fdf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.503786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a12c8b-c4ef-4694-a915-8001aca4fdf5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.506615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:31.527913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 324/653] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 26 05:30:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:31.528930 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a12c8b-c4ef-4694-a915-8001aca4fdf5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.548032 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:31.607108 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0a03f-9881-4a50-baca-d57135a78ad4 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.643528 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cd02d-2782-458d-b5d8-f7f02da43673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.644507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cd02d-2782-458d-b5d8-f7f02da43673 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.644933 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cd02d-2782-458d-b5d8-f7f02da43673 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.645394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cd02d-2782-458d-b5d8-f7f02da43673 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.646365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07cd02d-2782-458d-b5d8-f7f02da43673 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:31.646971 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07cd02d-2782-458d-b5d8-f7f02da43673 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.648561 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07cd02d-2782-458d-b5d8-f7f02da43673 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.662032 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a12c8b-c4ef-4694-a915-8001aca4fdf5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.667385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 330/654] 10.222.0.57 () {56 vars in 1137 bytes} [Mon Sep 26 05:30:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:31.668766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5026fa-338f-4d6c-b5bc-1d81f167a068 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.669110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5026fa-338f-4d6c-b5bc-1d81f167a068 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.669394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5026fa-338f-4d6c-b5bc-1d81f167a068 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.671895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 331/655] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:31.673012 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5026fa-338f-4d6c-b5bc-1d81f167a068 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.707605 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de909bca-5586-471b-8b76-9cdbd9255a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.708465 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de909bca-5586-471b-8b76-9cdbd9255a4e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.708911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de909bca-5586-471b-8b76-9cdbd9255a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.709393 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de909bca-5586-471b-8b76-9cdbd9255a4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.710490 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de909bca-5586-471b-8b76-9cdbd9255a4e None admin] RBAC: Adding query filter params (role.id=881210e567974dc0aa3043393c00a838, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=46edd6aba77c4404977b1c39e8442b2b) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:31.710914 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de909bca-5586-471b-8b76-9cdbd9255a4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.712470 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de909bca-5586-471b-8b76-9cdbd9255a4e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.718947 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 325/656] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:30:31 2022] GET /identity/v3/role_assignments?user.id=46edd6aba77c4404977b1c39e8442b2b&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=881210e567974dc0aa3043393c00a838 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:31.719810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de909bca-5586-471b-8b76-9cdbd9255a4e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:31.793803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:31.831651 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5026fa-338f-4d6c-b5bc-1d81f167a068 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.877932 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:31.899919 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd60c88-2216-4b15-9a7b-fbca24285a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.901058 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd60c88-2216-4b15-9a7b-fbca24285a45 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.901707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd60c88-2216-4b15-9a7b-fbca24285a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.902274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd60c88-2216-4b15-9a7b-fbca24285a45 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.903570 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd60c88-2216-4b15-9a7b-fbca24285a45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:31.904196 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd60c88-2216-4b15-9a7b-fbca24285a45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:31.904539 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de909bca-5586-471b-8b76-9cdbd9255a4e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:31.907677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd60c88-2216-4b15-9a7b-fbca24285a45 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:31.932058 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 332/657] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:31.933542 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72a20a-c3a1-48b7-868e-fad108e7cc58 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.933935 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72a20a-c3a1-48b7-868e-fad108e7cc58 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.934303 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72a20a-c3a1-48b7-868e-fad108e7cc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:31.954263 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-6c72a20a-c3a1-48b7-868e-fad108e7cc58 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:31.955615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6c72a20a-c3a1-48b7-868e-fad108e7cc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:31.975592 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6b9ed-b4cf-4857-a12f-d3a5a3d5490d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:31.976136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6b9ed-b4cf-4857-a12f-d3a5a3d5490d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:31.976410 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6b9ed-b4cf-4857-a12f-d3a5a3d5490d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:31.976679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6b9ed-b4cf-4857-a12f-d3a5a3d5490d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:32.053876 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 333/658] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:32.054751 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72a20a-c3a1-48b7-868e-fad108e7cc58 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:32.079220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6b9ed-b4cf-4857-a12f-d3a5a3d5490d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:32.082846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6b9ed-b4cf-4857-a12f-d3a5a3d5490d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:32.135818 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:32.172583 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72a20a-c3a1-48b7-868e-fad108e7cc58 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:32.212408 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b485745-9ccd-4b9e-9182-2694a6ed6807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:32.219615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b485745-9ccd-4b9e-9182-2694a6ed6807 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:32.220099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b485745-9ccd-4b9e-9182-2694a6ed6807 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:32.220450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b485745-9ccd-4b9e-9182-2694a6ed6807 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:32.258502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:32.297052 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 326/659] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:30:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:30:32.299102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6b9ed-b4cf-4857-a12f-d3a5a3d5490d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:32.308141 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b485745-9ccd-4b9e-9182-2694a6ed6807 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:32.309663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b485745-9ccd-4b9e-9182-2694a6ed6807 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:32.386207 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:32.429172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 334/660] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:30:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:30:32.430777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b485745-9ccd-4b9e-9182-2694a6ed6807 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:32.437796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6b9ed-b4cf-4857-a12f-d3a5a3d5490d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:32.481652 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b520e0f-63fd-4484-9b68-83c97c46d6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:32.482617 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b520e0f-63fd-4484-9b68-83c97c46d6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:32.482917 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b520e0f-63fd-4484-9b68-83c97c46d6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:32.483177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b520e0f-63fd-4484-9b68-83c97c46d6e3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:32.494910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b520e0f-63fd-4484-9b68-83c97c46d6e3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:32.496485 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b520e0f-63fd-4484-9b68-83c97c46d6e3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:32.509323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-0b520e0f-63fd-4484-9b68-83c97c46d6e3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 05:30:32.509323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:32.509323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:32.509323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:32.509323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:32.509323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:32.509323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:32.509323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:32.509323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:32.509323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:32.509323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:32.509323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:32.509323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:32.509323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:32.509323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:32.509323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:32.512173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:32.513177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:32.513177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:32.513177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:32.513177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:32.513177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:32.513177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:32.513177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:32.513177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:32.513177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 05:30:32.513177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:32.513177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 327/661] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:32.513177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b520e0f-63fd-4484-9b68-83c97c46d6e3 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:32.578900 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:32.580781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b485745-9ccd-4b9e-9182-2694a6ed6807 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:32.634374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d02e3-6f4f-4dc2-98f4-603f42090ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:32.635364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d02e3-6f4f-4dc2-98f4-603f42090ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:32.635823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d02e3-6f4f-4dc2-98f4-603f42090ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:32.636340 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d02e3-6f4f-4dc2-98f4-603f42090ec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:32.654347 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d02e3-6f4f-4dc2-98f4-603f42090ec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:32.656838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d02e3-6f4f-4dc2-98f4-603f42090ec6 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:32.672882 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b520e0f-63fd-4484-9b68-83c97c46d6e3 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:32.684069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-ca9d02e3-6f4f-4dc2-98f4-603f42090ec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:32.684069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:32.684069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:32.684069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:32.684069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:32.684069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:32.684069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:32.684069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:32.684069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:32.684069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:32.684069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:32.684069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:32.684069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:32.684069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:32.684069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:32.684069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:32.685257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:32.685257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:32.685257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:32.685257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:32.685257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:32.685257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:32.685257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:32.685257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:32.685257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:32.685257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:32.685257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:32.685257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:32.685257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:32.685257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:32.685257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:32.685257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:32.685257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:32.686277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:32.686277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:32.686277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:32.686277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 335/662] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:32.687694 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d02e3-6f4f-4dc2-98f4-603f42090ec6 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:32.719862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:32.724785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b1503-d542-47e2-b4ba-c1d662690cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:32.725630 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b1503-d542-47e2-b4ba-c1d662690cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:32.726100 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b1503-d542-47e2-b4ba-c1d662690cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:32.726262 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b1503-d542-47e2-b4ba-c1d662690cf1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:32.740662 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b1503-d542-47e2-b4ba-c1d662690cf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:32.746102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-3d8b1503-d542-47e2-b4ba-c1d662690cf1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:32.746102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:32.746102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:32.746102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:32.746102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:32.746102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:32.746102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:32.746102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:32.746102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:32.746102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:32.746102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:32.746102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:32.746102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:32.746102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:32.746102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:32.746102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:32.748206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:32.748206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:32.748206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:32.748206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:32.748206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:32.748206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:32.748206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:32.748206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:32.748206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:32.748206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:32.748206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:32.748206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:32.748206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:32.748206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:32.748206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:32.748206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:32.749795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:32.749795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:32.749795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:32.749795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:32.749795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:32.749795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:32.749795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:32.749795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:32.749795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:32.749795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:32.749795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:32.749795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:32.752202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 328/663] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:32.753173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b1503-d542-47e2-b4ba-c1d662690cf1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:32.837519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d02e3-6f4f-4dc2-98f4-603f42090ec6 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:32.873043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b1503-d542-47e2-b4ba-c1d662690cf1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:32.874735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:32.880663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef10936-a705-49bf-9ad3-1ab6b8dfb0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:32.883615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef10936-a705-49bf-9ad3-1ab6b8dfb0ed None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:32.883615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef10936-a705-49bf-9ad3-1ab6b8dfb0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:32.883615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef10936-a705-49bf-9ad3-1ab6b8dfb0ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:32.897551 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef10936-a705-49bf-9ad3-1ab6b8dfb0ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:32.900072 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef10936-a705-49bf-9ad3-1ab6b8dfb0ed None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:32.924259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-0ef10936-a705-49bf-9ad3-1ab6b8dfb0ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:32.924259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:32.924259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:32.924259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:32.924259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:32.924259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:32.924259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:32.924259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:32.924259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:32.924259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:32.924259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:32.924259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:32.924259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:32.924259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:32.924259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:32.924259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:32.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:32.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:32.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:32.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:32.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:32.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:32.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:32.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:32.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:32.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:32.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:32.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:32.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:32.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:32.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:32.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:32.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:32.926538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:32.926538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:32.926538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:32.926754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f11cd-d502-4a75-9ba2-2d8d16f952fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:32.926754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f11cd-d502-4a75-9ba2-2d8d16f952fa None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:32.926754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f11cd-d502-4a75-9ba2-2d8d16f952fa None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:32.926754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f11cd-d502-4a75-9ba2-2d8d16f952fa None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:32.927817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f11cd-d502-4a75-9ba2-2d8d16f952fa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:32.928319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f11cd-d502-4a75-9ba2-2d8d16f952fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:32.933912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 336/664] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:32.933912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef10936-a705-49bf-9ad3-1ab6b8dfb0ed None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:32.934193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f11cd-d502-4a75-9ba2-2d8d16f952fa None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:32.952510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 329/665] 10.222.0.57 () {56 vars in 1139 bytes} [Mon Sep 26 05:30:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:32.953355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f11cd-d502-4a75-9ba2-2d8d16f952fa None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:33.044681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:33.080307 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef10936-a705-49bf-9ad3-1ab6b8dfb0ed None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:33.084496 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f11cd-d502-4a75-9ba2-2d8d16f952fa None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:33.138473 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54a601-44d5-4f7e-946f-6f2e9dfaf9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:33.139603 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54a601-44d5-4f7e-946f-6f2e9dfaf9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:33.140096 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54a601-44d5-4f7e-946f-6f2e9dfaf9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:33.140616 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54a601-44d5-4f7e-946f-6f2e9dfaf9d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:33.141863 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54a601-44d5-4f7e-946f-6f2e9dfaf9d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:33.142444 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54a601-44d5-4f7e-946f-6f2e9dfaf9d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:33.143143 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f4771-9ad4-4a12-8131-36ffdbf2f9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:33.144150 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f4771-9ad4-4a12-8131-36ffdbf2f9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:33.144676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f4771-9ad4-4a12-8131-36ffdbf2f9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:33.145122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54a601-44d5-4f7e-946f-6f2e9dfaf9d4 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:33.145359 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f4771-9ad4-4a12-8131-36ffdbf2f9e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:33.146451 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f4771-9ad4-4a12-8131-36ffdbf2f9e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:33.147031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f4771-9ad4-4a12-8131-36ffdbf2f9e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:33.149350 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f4771-9ad4-4a12-8131-36ffdbf2f9e5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:33.168383 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 330/666] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:30:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:33.168383 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f4771-9ad4-4a12-8131-36ffdbf2f9e5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:33.172012 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 337/667] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:33.173100 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54a601-44d5-4f7e-946f-6f2e9dfaf9d4 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:33.205114 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:33.317402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f4771-9ad4-4a12-8131-36ffdbf2f9e5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:33.325233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54a601-44d5-4f7e-946f-6f2e9dfaf9d4 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:33.356850 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c67bd-40fb-424d-9680-904bf340d42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:33.357566 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c67bd-40fb-424d-9680-904bf340d42c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:33.358022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c67bd-40fb-424d-9680-904bf340d42c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:33.358450 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c67bd-40fb-424d-9680-904bf340d42c None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:33.358880 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c67bd-40fb-424d-9680-904bf340d42c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:33.359114 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c67bd-40fb-424d-9680-904bf340d42c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:33.360472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c67bd-40fb-424d-9680-904bf340d42c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:33.380508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 331/668] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:33.380508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c67bd-40fb-424d-9680-904bf340d42c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:33.381982 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c31deb-cdf0-4842-ad05-251c3afeaf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:33.382620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c31deb-cdf0-4842-ad05-251c3afeaf6a None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:33.382996 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c31deb-cdf0-4842-ad05-251c3afeaf6a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:33.383423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c31deb-cdf0-4842-ad05-251c3afeaf6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:33.387021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:33.492540 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:33.495288 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c31deb-cdf0-4842-ad05-251c3afeaf6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:33.496773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c31deb-cdf0-4842-ad05-251c3afeaf6a None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:33.514279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c67bd-40fb-424d-9680-904bf340d42c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:33.558916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb5402-f07c-4291-a24f-7eedfe96a27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:33.559758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb5402-f07c-4291-a24f-7eedfe96a27a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:33.559981 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb5402-f07c-4291-a24f-7eedfe96a27a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:33.560355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb5402-f07c-4291-a24f-7eedfe96a27a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:33.578254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb5402-f07c-4291-a24f-7eedfe96a27a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:33.580048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb5402-f07c-4291-a24f-7eedfe96a27a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:33.600005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-48bb5402-f07c-4291-a24f-7eedfe96a27a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:33.600005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:33.600005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:33.600005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:33.600005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:33.600005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:33.600005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:33.600005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:33.600005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:33.600005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:33.600005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:33.600005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:33.600005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:33.600005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:33.600005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:33.600005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:33.601186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:33.601186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:33.601186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:33.601186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:33.601186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:33.601186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:33.601186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:33.601186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:33.601186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:33.601186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:33.601186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:33.601186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:33.601186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:33.601186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:33.601186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:33.601186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:33.601186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:33.602230 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:33.602230 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:33.602230 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:33.607803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 332/669] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:33.609075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb5402-f07c-4291-a24f-7eedfe96a27a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:33.675349 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 338/670] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:30:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:30:33.675349 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c31deb-cdf0-4842-ad05-251c3afeaf6a None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:33.725330 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:33.760735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb5402-f07c-4291-a24f-7eedfe96a27a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:33.806248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d4750-40ca-4a0d-81bd-c4494486c72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:33.806897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d4750-40ca-4a0d-81bd-c4494486c72b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:33.807207 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d4750-40ca-4a0d-81bd-c4494486c72b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:33.807482 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d4750-40ca-4a0d-81bd-c4494486c72b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:33.810015 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:33.816807 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d4750-40ca-4a0d-81bd-c4494486c72b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:33.818404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d4750-40ca-4a0d-81bd-c4494486c72b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:33.832358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-848d4750-40ca-4a0d-81bd-c4494486c72b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:33.832358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:33.832358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:33.832358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:33.832358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:33.832358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:33.832358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:33.832358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:33.832358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:33.832358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:33.832358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:33.832358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:33.832358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:33.832358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:33.832358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:33.834185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:33.834185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:33.834185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:33.834185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:33.834185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:33.834185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:33.834185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:33.834185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:33.834185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:33.834185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:33.834185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:33.834185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:33.834185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:33.834185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:33.834185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:33.834185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:33.834185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:33.836748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:33.836748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:33.836748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:33.836748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:33.836748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:33.836748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:33.836748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:33.836748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:33.836748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 333/671] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:33.836748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d4750-40ca-4a0d-81bd-c4494486c72b None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:33.860749 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c31deb-cdf0-4842-ad05-251c3afeaf6a None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:33.921676 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72680d0f-b272-4c64-81e3-e06da0618be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:33.923018 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72680d0f-b272-4c64-81e3-e06da0618be1 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:33.923653 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72680d0f-b272-4c64-81e3-e06da0618be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:33.924154 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72680d0f-b272-4c64-81e3-e06da0618be1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:33.939083 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72680d0f-b272-4c64-81e3-e06da0618be1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:33.941541 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72680d0f-b272-4c64-81e3-e06da0618be1 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:33.964743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-72680d0f-b272-4c64-81e3-e06da0618be1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:33.964743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:33.964743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:33.964743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:33.964743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:33.964743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:33.964743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:33.964743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:33.964743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:33.964743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:33.964743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:33.964743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:33.964743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:33.964743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:33.964743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:33.966287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:33.966287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:33.966287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:33.966287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:33.966287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:33.966287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:33.966287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:33.966287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:33.966287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:33.966287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:33.966287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:33.966287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:33.966287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:33.966287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:33.966287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:33.966287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:33.966287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:33.967350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:33.967350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:33.967350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:33.967350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:33.967350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:33.967350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:33.967350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:33.967350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:33.967350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 339/672] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:33.968028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72680d0f-b272-4c64-81e3-e06da0618be1 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:33.990917 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d4750-40ca-4a0d-81bd-c4494486c72b None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:34.032541 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f45d0-75a8-4c42-b662-97bc6686d218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:34.033227 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f45d0-75a8-4c42-b662-97bc6686d218 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:34.033515 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f45d0-75a8-4c42-b662-97bc6686d218 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:34.033787 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f45d0-75a8-4c42-b662-97bc6686d218 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:34.034446 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f45d0-75a8-4c42-b662-97bc6686d218 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:34.034747 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f45d0-75a8-4c42-b662-97bc6686d218 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:34.036127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f45d0-75a8-4c42-b662-97bc6686d218 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:34.050257 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 334/673] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:34.050997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f45d0-75a8-4c42-b662-97bc6686d218 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:34.110095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:34.119426 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:34.125525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72680d0f-b272-4c64-81e3-e06da0618be1 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:34.180791 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3792de-4957-44c9-810f-aa36176c7570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:34.181618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3792de-4957-44c9-810f-aa36176c7570 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:34.181999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3792de-4957-44c9-810f-aa36176c7570 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:34.182428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3792de-4957-44c9-810f-aa36176c7570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:34.195414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f45d0-75a8-4c42-b662-97bc6686d218 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:34.195681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3792de-4957-44c9-810f-aa36176c7570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:34.197100 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3792de-4957-44c9-810f-aa36176c7570 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:34.214774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-2f3792de-4957-44c9-810f-aa36176c7570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:34.214774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:34.214774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:34.214774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:34.214774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:34.214774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:34.214774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:34.214774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:34.214774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:34.214774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:34.214774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:34.214774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:34.214774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:34.214774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:34.214774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:34.214774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:34.218329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:34.218329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:34.218329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:34.218329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:34.218329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:34.218329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:34.218329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:34.218329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:34.218329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:34.218329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:34.218329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:34.218329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:34.218329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:34.218329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:34.218329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:34.218329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:34.218329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:34.220050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:34.220050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:34.220050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:34.220050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 340/674] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:34.220050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3792de-4957-44c9-810f-aa36176c7570 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:34.254033 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ec573-f0f3-4c39-afe5-2481f9b8a03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:34.254654 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ec573-f0f3-4c39-afe5-2481f9b8a03d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:34.255002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ec573-f0f3-4c39-afe5-2481f9b8a03d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:34.255345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ec573-f0f3-4c39-afe5-2481f9b8a03d None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:34.256296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ec573-f0f3-4c39-afe5-2481f9b8a03d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:34.256690 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ec573-f0f3-4c39-afe5-2481f9b8a03d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:34.258301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ec573-f0f3-4c39-afe5-2481f9b8a03d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:34.281741 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 335/675] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:34.283452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ea57e-6628-40b9-b753-b5473a7ffff8 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:34.283873 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ea57e-6628-40b9-b753-b5473a7ffff8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:34.284262 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ea57e-6628-40b9-b753-b5473a7ffff8 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:34.286040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 336/676] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:34.287291 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ea57e-6628-40b9-b753-b5473a7ffff8 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:34.357929 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3792de-4957-44c9-810f-aa36176c7570 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:34.400970 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7a9dc-0a1f-4516-a76d-8619d062a7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:34.401232 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7a9dc-0a1f-4516-a76d-8619d062a7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:34.401497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7a9dc-0a1f-4516-a76d-8619d062a7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:34.401880 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7a9dc-0a1f-4516-a76d-8619d062a7f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:34.402635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7a9dc-0a1f-4516-a76d-8619d062a7f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:34.402937 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7a9dc-0a1f-4516-a76d-8619d062a7f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:34.404408 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7a9dc-0a1f-4516-a76d-8619d062a7f6 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:34.419081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 341/677] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:34.420198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7a9dc-0a1f-4516-a76d-8619d062a7f6 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:34.427643 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ea57e-6628-40b9-b753-b5473a7ffff8 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:34.441710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:34.450996 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:34.487886 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99014cd-bce7-417e-8a5b-28416cac28d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:34.489068 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99014cd-bce7-417e-8a5b-28416cac28d9 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:34.489589 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99014cd-bce7-417e-8a5b-28416cac28d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:34.490049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99014cd-bce7-417e-8a5b-28416cac28d9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:34.506437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99014cd-bce7-417e-8a5b-28416cac28d9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:34.508947 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99014cd-bce7-417e-8a5b-28416cac28d9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:34.527525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-d99014cd-bce7-417e-8a5b-28416cac28d9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:30:34.527525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:34.527525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:34.527525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:34.527525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:34.527525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:34.527525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:34.527525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:34.527525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:34.527525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:34.527525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:34.527525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:34.527525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:34.527525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:34.527525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:34.527525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:34.529701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:34.531764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:34.531764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:34.531764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:34.531764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:34.531764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:34.531764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:34.531764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:34.531764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:34.531764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:30:34.531764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:34.531764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 337/678] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:30:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:34.531764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99014cd-bce7-417e-8a5b-28416cac28d9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:34.577263 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7a9dc-0a1f-4516-a76d-8619d062a7f6 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:34.634244 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311391b-a8a1-4145-9715-b42c3c34cef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:34.634859 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311391b-a8a1-4145-9715-b42c3c34cef0 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:34.635133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311391b-a8a1-4145-9715-b42c3c34cef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:34.635398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311391b-a8a1-4145-9715-b42c3c34cef0 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:34.636184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311391b-a8a1-4145-9715-b42c3c34cef0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:34.636441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311391b-a8a1-4145-9715-b42c3c34cef0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:34.637958 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311391b-a8a1-4145-9715-b42c3c34cef0 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:34.651195 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 342/679] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:34.652812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda776b0-2d26-4b91-8899-5398d74c0e6a None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:34.652812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda776b0-2d26-4b91-8899-5398d74c0e6a None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:34.653120 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda776b0-2d26-4b91-8899-5398d74c0e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:34.672142 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-dda776b0-2d26-4b91-8899-5398d74c0e6a None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:34.673394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-dda776b0-2d26-4b91-8899-5398d74c0e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:34.676130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99014cd-bce7-417e-8a5b-28416cac28d9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:34.729318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b090a-cd87-4767-aed8-412a0b822a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:34.731696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b090a-cd87-4767-aed8-412a0b822a74 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:34.731696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b090a-cd87-4767-aed8-412a0b822a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:34.731696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b090a-cd87-4767-aed8-412a0b822a74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:34.731696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b090a-cd87-4767-aed8-412a0b822a74 None admin] RBAC: Adding query filter params (role.id=881210e567974dc0aa3043393c00a838, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=92556048cbd04c409c3f4b6fd645627f) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:34.732230 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b090a-cd87-4767-aed8-412a0b822a74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:34.733578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94b090a-cd87-4767-aed8-412a0b822a74 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:34.745201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 338/680] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:30:34 2022] GET /identity/v3/role_assignments?user.id=92556048cbd04c409c3f4b6fd645627f&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=881210e567974dc0aa3043393c00a838 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:34.745201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b090a-cd87-4767-aed8-412a0b822a74 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:34.759161 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:34.780564 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 343/681] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:34.782062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda776b0-2d26-4b91-8899-5398d74c0e6a None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:34.916972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda776b0-2d26-4b91-8899-5398d74c0e6a None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:34.939674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:34.968521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8f1d2-3752-4de9-aca6-a27d4f991c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:34.969951 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8f1d2-3752-4de9-aca6-a27d4f991c81 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:34.971316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8f1d2-3752-4de9-aca6-a27d4f991c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:34.972193 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8f1d2-3752-4de9-aca6-a27d4f991c81 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:34.973214 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8f1d2-3752-4de9-aca6-a27d4f991c81 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:34.973755 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8f1d2-3752-4de9-aca6-a27d4f991c81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:34.976878 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8f1d2-3752-4de9-aca6-a27d4f991c81 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:35.000941 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 344/682] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Sep 26 05:30:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:35.003301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8f1d2-3752-4de9-aca6-a27d4f991c81 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:35.083789 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b090a-cd87-4767-aed8-412a0b822a74 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:35.107322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:35.140083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2de36-815d-4455-8696-c9e52a01a504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:35.141339 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2de36-815d-4455-8696-c9e52a01a504 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:35.141615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2de36-815d-4455-8696-c9e52a01a504 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:35.141884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2de36-815d-4455-8696-c9e52a01a504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:35.142625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2de36-815d-4455-8696-c9e52a01a504 None admin] RBAC: Adding query filter params (role.id=881210e567974dc0aa3043393c00a838, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=90c0fabecff2457e931c8c1c36681197) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:35.143354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2de36-815d-4455-8696-c9e52a01a504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:35.145177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2de36-815d-4455-8696-c9e52a01a504 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:35.154895 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 339/683] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:30:34 2022] GET /identity/v3/role_assignments?user.id=90c0fabecff2457e931c8c1c36681197&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=881210e567974dc0aa3043393c00a838 => generated 608 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:35.155976 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2de36-815d-4455-8696-c9e52a01a504 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:35.189335 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8f1d2-3752-4de9-aca6-a27d4f991c81 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:35.326036 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d521b6-440b-4869-ab0a-c68fce52e547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:35.330613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d521b6-440b-4869-ab0a-c68fce52e547 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:35.330613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d521b6-440b-4869-ab0a-c68fce52e547 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:35.330613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d521b6-440b-4869-ab0a-c68fce52e547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:35.356761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d521b6-440b-4869-ab0a-c68fce52e547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:35.356761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d521b6-440b-4869-ab0a-c68fce52e547 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:35.397212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-f9d521b6-440b-4869-ab0a-c68fce52e547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:35.397212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:35.397212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:35.397212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:35.397212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:35.397212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:35.397212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:35.397212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:35.397212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:35.397212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:35.397212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:35.397212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:35.397212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:35.397212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:35.397212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:35.398727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:35.398727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:35.398727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:35.398727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:35.398727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:35.398727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:35.398727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:35.398727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:35.398727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:35.398727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:35.398727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:35.398727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:35.398727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:35.398727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:35.398727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:35.398727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:35.398727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:35.400216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:35.400216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:35.400216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:35.400216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:35.400216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:35.400216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:35.400216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:35.400216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:35.403974 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 345/684] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:35.403974 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d521b6-440b-4869-ab0a-c68fce52e547 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:35.519105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2de36-815d-4455-8696-c9e52a01a504 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:35.521994 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:35.580096 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d521b6-440b-4869-ab0a-c68fce52e547 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:35.587947 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8126e28-b467-42c4-9720-3c914aee9102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:35.588979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8126e28-b467-42c4-9720-3c914aee9102 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:35.589536 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8126e28-b467-42c4-9720-3c914aee9102 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:35.591716 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8126e28-b467-42c4-9720-3c914aee9102 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:35.606122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:35.646366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7889fb-05dd-48ea-8cfa-9f144eebae28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:35.647322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7889fb-05dd-48ea-8cfa-9f144eebae28 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:35.648671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7889fb-05dd-48ea-8cfa-9f144eebae28 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:35.649109 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7889fb-05dd-48ea-8cfa-9f144eebae28 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:35.651703 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7889fb-05dd-48ea-8cfa-9f144eebae28 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:35.653398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7889fb-05dd-48ea-8cfa-9f144eebae28 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:35.674464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-dd7889fb-05dd-48ea-8cfa-9f144eebae28 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:30:35.674464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:35.674464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:30:35.674464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:30:35.674464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:30:35.674464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:30:35.674464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:30:35.674464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:30:35.674464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:30:35.674464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:35.674464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:30:35.674464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:35.674464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:35.674464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:35.674464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:35.674464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:35.674464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:35.675824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:35.675824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:35.675824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:35.675824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:35.675824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:35.675824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:35.675824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:30:35.675824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:30:35.675824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:30:35.675824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:30:35.675824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:35.675824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:35.675824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:35.675824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:35.675824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:35.675824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:35.676886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:35.676886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:35.676886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:35.676886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:35.676886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:35.676886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:35.676886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:35.676886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:35.676886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:35.676886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:35.676886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:30:35.676886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:30:35.676886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:30:35.676886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:35.690654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 346/685] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Sep 26 05:30:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:35.691905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7889fb-05dd-48ea-8cfa-9f144eebae28 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:35.756270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8126e28-b467-42c4-9720-3c914aee9102 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:35.760108 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8126e28-b467-42c4-9720-3c914aee9102 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:35.889232 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7889fb-05dd-48ea-8cfa-9f144eebae28 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:35.945771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f266f11-b10b-431d-86b7-229dd8e7da84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:35.946761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f266f11-b10b-431d-86b7-229dd8e7da84 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:35.947209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f266f11-b10b-431d-86b7-229dd8e7da84 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:35.947653 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f266f11-b10b-431d-86b7-229dd8e7da84 None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:35.948631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f266f11-b10b-431d-86b7-229dd8e7da84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:35.951307 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f266f11-b10b-431d-86b7-229dd8e7da84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:35.953091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f266f11-b10b-431d-86b7-229dd8e7da84 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:35.978578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 340/686] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:30:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 823 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:30:35.979855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8126e28-b467-42c4-9720-3c914aee9102 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:35.984648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 347/687] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:35.988418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f266f11-b10b-431d-86b7-229dd8e7da84 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:35.990212 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:36.096080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:36.246428 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8126e28-b467-42c4-9720-3c914aee9102 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:36.261292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f266f11-b10b-431d-86b7-229dd8e7da84 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:36.335916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721643f-7908-45a3-b4c5-3e5bea833d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:36.336548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721643f-7908-45a3-b4c5-3e5bea833d6d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:36.336826 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721643f-7908-45a3-b4c5-3e5bea833d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:36.337091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721643f-7908-45a3-b4c5-3e5bea833d6d None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:36.337797 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721643f-7908-45a3-b4c5-3e5bea833d6d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:36.338097 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721643f-7908-45a3-b4c5-3e5bea833d6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:36.339553 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721643f-7908-45a3-b4c5-3e5bea833d6d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:36.346951 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6ad1a-15fe-4edd-a83b-28cc4a59de54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:36.347823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6ad1a-15fe-4edd-a83b-28cc4a59de54 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:36.348882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6ad1a-15fe-4edd-a83b-28cc4a59de54 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:36.349210 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6ad1a-15fe-4edd-a83b-28cc4a59de54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:36.356116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 341/688] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Sep 26 05:30:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:36.357105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721643f-7908-45a3-b4c5-3e5bea833d6d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:36.380635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6ad1a-15fe-4edd-a83b-28cc4a59de54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:36.382244 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6ad1a-15fe-4edd-a83b-28cc4a59de54 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:36.420962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-b1f6ad1a-15fe-4edd-a83b-28cc4a59de54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:36.420962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:36.420962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:36.420962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:36.420962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:36.420962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:36.420962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:36.420962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:36.420962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:36.420962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:36.420962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:36.420962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:36.420962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:36.420962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:36.420962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:36.420962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:36.422038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:36.422038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:36.422038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:36.422038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:36.422038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:36.422038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:36.422038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:36.422038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:36.422038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:36.422038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:36.422038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:36.422038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:36.422038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:36.422038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:36.422038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:36.422038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:36.422038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:36.422950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:36.422950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:36.422950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:36.426389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 348/689] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 438 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:36.427179 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6ad1a-15fe-4edd-a83b-28cc4a59de54 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:36.510502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:36.587542 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721643f-7908-45a3-b4c5-3e5bea833d6d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:36.620486 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:36.635057 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6ad1a-15fe-4edd-a83b-28cc4a59de54 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:36.654693 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915222d-d9f5-488b-8d17-8fdee8936aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:36.655744 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915222d-d9f5-488b-8d17-8fdee8936aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:36.656233 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915222d-d9f5-488b-8d17-8fdee8936aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:36.656719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915222d-d9f5-488b-8d17-8fdee8936aa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:36.659554 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915222d-d9f5-488b-8d17-8fdee8936aa8 None admin] RBAC: Adding query filter params (role.id=881210e567974dc0aa3043393c00a838, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=6bf0f594ce7646b7b6ddb8a8553509d7) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:36.660201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915222d-d9f5-488b-8d17-8fdee8936aa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:36.661984 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915222d-d9f5-488b-8d17-8fdee8936aa8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:36.670969 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 342/690] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:30:36 2022] GET /identity/v3/role_assignments?user.id=6bf0f594ce7646b7b6ddb8a8553509d7&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=881210e567974dc0aa3043393c00a838 => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:36.678005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915222d-d9f5-488b-8d17-8fdee8936aa8 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:36.694479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470898c-2289-41e9-a065-bd08135542be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:36.695330 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470898c-2289-41e9-a065-bd08135542be None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:36.695611 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470898c-2289-41e9-a065-bd08135542be None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:36.695878 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470898c-2289-41e9-a065-bd08135542be None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:36.696767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470898c-2289-41e9-a065-bd08135542be None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:36.697063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470898c-2289-41e9-a065-bd08135542be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:36.698554 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470898c-2289-41e9-a065-bd08135542be None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:36.712905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 349/691] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Sep 26 05:30:36 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:36.830655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915222d-d9f5-488b-8d17-8fdee8936aa8 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:36.875078 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df750a-7d5a-40a4-afa2-68f4e25909cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:36.876147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df750a-7d5a-40a4-afa2-68f4e25909cb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:36.876828 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df750a-7d5a-40a4-afa2-68f4e25909cb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:36.877378 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df750a-7d5a-40a4-afa2-68f4e25909cb None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:36.878401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df750a-7d5a-40a4-afa2-68f4e25909cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:36.878953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df750a-7d5a-40a4-afa2-68f4e25909cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:36.880667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df750a-7d5a-40a4-afa2-68f4e25909cb None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:36.883148 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:36.897970 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 343/692] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:36.904815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470898c-2289-41e9-a065-bd08135542be None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:37.066819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470898c-2289-41e9-a065-bd08135542be None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:37.124676 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38de37-41c4-4be8-9e02-3523d6956ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:37.125509 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38de37-41c4-4be8-9e02-3523d6956ddf None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:37.125867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38de37-41c4-4be8-9e02-3523d6956ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:37.126246 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38de37-41c4-4be8-9e02-3523d6956ddf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:37.142539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38de37-41c4-4be8-9e02-3523d6956ddf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:37.144123 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38de37-41c4-4be8-9e02-3523d6956ddf None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:37.158290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-7c38de37-41c4-4be8-9e02-3523d6956ddf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:37.158290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:37.158290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:37.158290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:37.158290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:37.158290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:37.158290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:37.158290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:37.158290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:37.158290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:37.158290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:37.158290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:37.158290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:37.158290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:37.158290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:37.159287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:37.159287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:37.159287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:37.159287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:37.159287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:37.159287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:37.159287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:37.159287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:37.159287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:37.159287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:37.159287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:37.159287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:37.159287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:37.159287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:37.159287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:37.159287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:37.159287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:37.160187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:37.160187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:37.160187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:37.160187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:37.160187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:37.160187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:37.160187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:37.160187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:37.165171 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 350/693] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:37.166130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df750a-7d5a-40a4-afa2-68f4e25909cb None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:37.407487 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df750a-7d5a-40a4-afa2-68f4e25909cb None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:37.499505 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cc92a-4020-40a1-aa4e-45eaafa01aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:37.500148 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cc92a-4020-40a1-aa4e-45eaafa01aaa None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:37.500423 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cc92a-4020-40a1-aa4e-45eaafa01aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:37.500692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cc92a-4020-40a1-aa4e-45eaafa01aaa None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:37.501449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cc92a-4020-40a1-aa4e-45eaafa01aaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:37.501756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cc92a-4020-40a1-aa4e-45eaafa01aaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:37.503249 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cc92a-4020-40a1-aa4e-45eaafa01aaa None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:37.532960 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 344/694] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:37.540024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38de37-41c4-4be8-9e02-3523d6956ddf None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:37.579514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:37.725256 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38de37-41c4-4be8-9e02-3523d6956ddf None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:37.769278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5a592-2d41-477d-b302-9b1fc61882ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:37.769278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5a592-2d41-477d-b302-9b1fc61882ae None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:37.769278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5a592-2d41-477d-b302-9b1fc61882ae None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:37.769278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5a592-2d41-477d-b302-9b1fc61882ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:37.769726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5a592-2d41-477d-b302-9b1fc61882ae None admin] RBAC: Adding query filter params (role.id=881210e567974dc0aa3043393c00a838, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=00f258aba84d4dd59ebe8de973ac65c6) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:37.769726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5a592-2d41-477d-b302-9b1fc61882ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:37.769871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5a592-2d41-477d-b302-9b1fc61882ae None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:37.781283 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 351/695] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:30:37 2022] GET /identity/v3/role_assignments?user.id=00f258aba84d4dd59ebe8de973ac65c6&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=881210e567974dc0aa3043393c00a838 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:30:37.993437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13401c-123b-45d7-b381-905bb36e270a None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:37.993996 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13401c-123b-45d7-b381-905bb36e270a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:37.993996 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13401c-123b-45d7-b381-905bb36e270a None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:37.995828 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 345/696] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:38.005900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28490450-6ae6-40f3-b506-2ef7b7a340e0 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:38.006231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28490450-6ae6-40f3-b506-2ef7b7a340e0 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:38.006455 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28490450-6ae6-40f3-b506-2ef7b7a340e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:38.028092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-28490450-6ae6-40f3-b506-2ef7b7a340e0 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:38.029402 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-28490450-6ae6-40f3-b506-2ef7b7a340e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:38.121281 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 352/697] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:38.219187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a468b-ba70-4c65-9e56-e4ed083e1e0a None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:38.219187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a468b-ba70-4c65-9e56-e4ed083e1e0a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:38.219187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a468b-ba70-4c65-9e56-e4ed083e1e0a None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:38.226604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 346/698] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:38.231001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59054db7-6d21-488e-9a51-e783c560edab None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:38.231471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59054db7-6d21-488e-9a51-e783c560edab None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:38.232253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59054db7-6d21-488e-9a51-e783c560edab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:38.287119 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-59054db7-6d21-488e-9a51-e783c560edab None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:38.289069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-59054db7-6d21-488e-9a51-e783c560edab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:38.297403 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:38.456438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a468b-ba70-4c65-9e56-e4ed083e1e0a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:38.465200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 353/699] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:38.600784 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:38.657895 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a468b-ba70-4c65-9e56-e4ed083e1e0a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:38.715165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc748de1-9271-4aba-86ca-4bfbe111f1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:38.715834 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc748de1-9271-4aba-86ca-4bfbe111f1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:38.716111 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc748de1-9271-4aba-86ca-4bfbe111f1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:38.716570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc748de1-9271-4aba-86ca-4bfbe111f1a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:38.731195 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc748de1-9271-4aba-86ca-4bfbe111f1a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:38.731958 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc748de1-9271-4aba-86ca-4bfbe111f1a7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:38.748711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-cc748de1-9271-4aba-86ca-4bfbe111f1a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:38.748711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:38.748711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:38.748711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:38.748711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:38.748711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:38.748711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:38.748711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:38.748711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:38.748711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:38.748711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:38.748711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:38.748711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:38.748711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:38.748711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:38.748711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:38.750598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:38.750598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:38.750598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:38.750598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:38.750598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:38.750598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:38.750598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:38.750598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:38.750598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:38.750598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:38.750598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:38.750598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:38.750598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:38.750598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:38.750598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:38.750598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:38.750598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:38.752302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:38.752302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:38.752302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:38.755674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 347/700] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:30:38.758643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59054db7-6d21-488e-9a51-e783c560edab None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:38.939072 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc748de1-9271-4aba-86ca-4bfbe111f1a7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:39.057244 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59054db7-6d21-488e-9a51-e783c560edab None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:39.125922 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b81f7c-502e-422a-aaed-091f98118f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:39.127750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b81f7c-502e-422a-aaed-091f98118f14 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:39.128205 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b81f7c-502e-422a-aaed-091f98118f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:39.128706 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b81f7c-502e-422a-aaed-091f98118f14 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:39.129641 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b81f7c-502e-422a-aaed-091f98118f14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:39.131031 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b81f7c-502e-422a-aaed-091f98118f14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:39.133042 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b81f7c-502e-422a-aaed-091f98118f14 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:39.142423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:39.164470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 354/701] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:39.166001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b81f7c-502e-422a-aaed-091f98118f14 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:39.247263 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc748de1-9271-4aba-86ca-4bfbe111f1a7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:39.249099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:39.325308 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd67242-8616-4b24-943f-bbe80db7d91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:39.325971 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd67242-8616-4b24-943f-bbe80db7d91c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:39.326246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd67242-8616-4b24-943f-bbe80db7d91c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:39.326519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd67242-8616-4b24-943f-bbe80db7d91c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:39.354170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd67242-8616-4b24-943f-bbe80db7d91c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:39.359739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-9dd67242-8616-4b24-943f-bbe80db7d91c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:30:39.359739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:39.359739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:39.359739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:39.359739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:39.359739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:39.359739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:39.359739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:39.359739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:39.359739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:39.359739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:39.359739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:39.359739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:39.359739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:39.359739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:39.359739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:39.361596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:39.361596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:39.361596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:39.361596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:39.361596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:39.361596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:39.361596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:39.361596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:39.361596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:39.361596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:39.361596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:39.361596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:39.361596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:39.361596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:39.361596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:39.361596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:39.363284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:39.363284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:39.363284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:39.363284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:39.363284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:39.363284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:39.363284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:39.363284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:39.363284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:39.363284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:39.363284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:30:39.363284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:39.375843 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 348/702] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:30:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 437 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:39.376830 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd67242-8616-4b24-943f-bbe80db7d91c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:39.401649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b81f7c-502e-422a-aaed-091f98118f14 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:39.512573 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f247d-4ce1-46ba-a4eb-073b96a20ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:39.513107 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f247d-4ce1-46ba-a4eb-073b96a20ebe None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:39.513427 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f247d-4ce1-46ba-a4eb-073b96a20ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:39.513675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f247d-4ce1-46ba-a4eb-073b96a20ebe None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:39.514344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f247d-4ce1-46ba-a4eb-073b96a20ebe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:39.515810 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f247d-4ce1-46ba-a4eb-073b96a20ebe None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:39.595109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd67242-8616-4b24-943f-bbe80db7d91c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:39.617473 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 355/703] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:30:39 2022] POST /identity/v3/users => generated 254 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:39.675489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc49d6-3561-45fb-9ada-63dee38513f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:39.676118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc49d6-3561-45fb-9ada-63dee38513f7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:39.676404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc49d6-3561-45fb-9ada-63dee38513f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:39.676821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc49d6-3561-45fb-9ada-63dee38513f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:39.681748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc49d6-3561-45fb-9ada-63dee38513f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:39.682179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc49d6-3561-45fb-9ada-63dee38513f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:39.683668 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc49d6-3561-45fb-9ada-63dee38513f7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:39.702178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 349/704] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:30:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:39.708988 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f247d-4ce1-46ba-a4eb-073b96a20ebe None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:39.933500 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:40.037103 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f247d-4ce1-46ba-a4eb-073b96a20ebe None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:40.106355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9e6ff-2924-4ea3-8e91-b826f99b2c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:40.107385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9e6ff-2924-4ea3-8e91-b826f99b2c51 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.107870 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9e6ff-2924-4ea3-8e91-b826f99b2c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.108402 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9e6ff-2924-4ea3-8e91-b826f99b2c51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.137466 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9e6ff-2924-4ea3-8e91-b826f99b2c51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:40.142809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9e6ff-2924-4ea3-8e91-b826f99b2c51 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:40.166681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-7ca9e6ff-2924-4ea3-8e91-b826f99b2c51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:40.166681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:40.166681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:40.166681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:40.166681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:40.166681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:40.166681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:40.166681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:40.166681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:40.166681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:40.166681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:40.166681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:40.166681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:40.166681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:40.166681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:40.166681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:40.167819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:40.167819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:40.167819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:40.167819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:40.167819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:40.167819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:40.167819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:40.167819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:40.167819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:40.167819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:40.167819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:40.167819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:40.167819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:40.167819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:40.167819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:40.167819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:40.167819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:40.168739 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:40.168739 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:40.168739 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:40.169354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 356/705] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 461 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:40.174991 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc49d6-3561-45fb-9ada-63dee38513f7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:40.223920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:40.334887 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc49d6-3561-45fb-9ada-63dee38513f7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:40.382774 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dabfe-246e-43ca-bc51-c789bae79327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:40.383437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dabfe-246e-43ca-bc51-c789bae79327 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.383708 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dabfe-246e-43ca-bc51-c789bae79327 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.383972 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dabfe-246e-43ca-bc51-c789bae79327 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.384654 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02dabfe-246e-43ca-bc51-c789bae79327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:40.384952 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02dabfe-246e-43ca-bc51-c789bae79327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:40.386503 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02dabfe-246e-43ca-bc51-c789bae79327 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:40.410185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 350/706] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:40.420781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9e6ff-2924-4ea3-8e91-b826f99b2c51 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:40.518920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f17f68-3586-49b5-9dea-43283e0ec3ba None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.519111 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f17f68-3586-49b5-9dea-43283e0ec3ba None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.519362 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f17f68-3586-49b5-9dea-43283e0ec3ba None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.520797 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 351/707] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:40.535451 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd19cf-9f43-4c22-8fb3-08739afa5bd5 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.535945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd19cf-9f43-4c22-8fb3-08739afa5bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.536424 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd19cf-9f43-4c22-8fb3-08739afa5bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.601325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-99bd19cf-9f43-4c22-8fb3-08739afa5bd5 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:40.603306 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-99bd19cf-9f43-4c22-8fb3-08739afa5bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:40.690366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9e6ff-2924-4ea3-8e91-b826f99b2c51 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:40.713899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:40.763801 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 352/708] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:40.767911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cc190-c15f-4424-b604-d5de62919ba9 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.767911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cc190-c15f-4424-b604-d5de62919ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.767911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cc190-c15f-4424-b604-d5de62919ba9 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.771943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 353/709] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:40.778543 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a8715-3b1d-429d-9100-725fd0e8c344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:40.780700 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72dd55-a450-4def-aab4-a50609779222 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.781254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72dd55-a450-4def-aab4-a50609779222 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.781695 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72dd55-a450-4def-aab4-a50609779222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.785529 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a8715-3b1d-429d-9100-725fd0e8c344 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:40.786062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a8715-3b1d-429d-9100-725fd0e8c344 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:40.786498 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a8715-3b1d-429d-9100-725fd0e8c344 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:40.800354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a8715-3b1d-429d-9100-725fd0e8c344 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:40.807069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542a8715-3b1d-429d-9100-725fd0e8c344 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:40.822328 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-6d72dd55-a450-4def-aab4-a50609779222 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:40.823647 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6d72dd55-a450-4def-aab4-a50609779222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:40.828241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-542a8715-3b1d-429d-9100-725fd0e8c344 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:30:40.828241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:40.828241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:40.828241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:40.828241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:40.828241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:40.828241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:40.828241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:40.828241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:40.828241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:40.828241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:40.828241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:40.828241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:40.828241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:40.828241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:40.828241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:40.829271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:40.830206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:40.830206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:40.830206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:40.830206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:40.830206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:40.830206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:40.830206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:40.830206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:40.830206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:30:40.830206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:40.835939 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 357/710] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 26 05:30:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:40.839820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a8715-3b1d-429d-9100-725fd0e8c344 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:40.859604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:40.977167 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a8715-3b1d-429d-9100-725fd0e8c344 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:41.016425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 354/711] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:41.058015 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad4644-590f-4539-b4ab-03c2d7cfb7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:41.059028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad4644-590f-4539-b4ab-03c2d7cfb7fd None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:41.059502 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad4644-590f-4539-b4ab-03c2d7cfb7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:41.060041 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad4644-590f-4539-b4ab-03c2d7cfb7fd None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:41.060993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad4644-590f-4539-b4ab-03c2d7cfb7fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:41.061526 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad4644-590f-4539-b4ab-03c2d7cfb7fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:41.064082 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad4644-590f-4539-b4ab-03c2d7cfb7fd None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:41.114455 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 358/712] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 26 05:30:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:41.121058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72dd55-a450-4def-aab4-a50609779222 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:41.212661 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad4644-590f-4539-b4ab-03c2d7cfb7fd None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:41.271249 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:41.313231 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72dd55-a450-4def-aab4-a50609779222 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:41.383229 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967a701-f939-43f1-82c4-33fe2651b2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:41.383229 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967a701-f939-43f1-82c4-33fe2651b2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:41.383229 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967a701-f939-43f1-82c4-33fe2651b2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:41.384306 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967a701-f939-43f1-82c4-33fe2651b2b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:41.398880 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad4644-590f-4539-b4ab-03c2d7cfb7fd None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:41.411004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:41.445757 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa72c7f-6552-4175-b1e6-e49b345242bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:41.446388 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa72c7f-6552-4175-b1e6-e49b345242bb None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:41.446735 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa72c7f-6552-4175-b1e6-e49b345242bb None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:41.447008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa72c7f-6552-4175-b1e6-e49b345242bb None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:41.448725 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa72c7f-6552-4175-b1e6-e49b345242bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:41.450188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa72c7f-6552-4175-b1e6-e49b345242bb None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:41.474779 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 359/713] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 26 05:30:41 2022] POST /identity/v3/services => generated 251 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:41.485562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c5823-97cb-44e6-a287-eb8d19b4f9ce None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:41.485562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c5823-97cb-44e6-a287-eb8d19b4f9ce None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:41.485562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c5823-97cb-44e6-a287-eb8d19b4f9ce None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:41.485562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 360/714] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:41.485562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c5823-97cb-44e6-a287-eb8d19b4f9ce None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:41.496714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967a701-f939-43f1-82c4-33fe2651b2b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:41.498900 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967a701-f939-43f1-82c4-33fe2651b2b0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:41.615323 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:41.644563 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 355/715] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:30:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:30:41.646391 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775d591-0403-422c-9e3a-76e7872366df None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:41.646911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775d591-0403-422c-9e3a-76e7872366df None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:41.647435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775d591-0403-422c-9e3a-76e7872366df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:41.701522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8775d591-0403-422c-9e3a-76e7872366df None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:41.702590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c5823-97cb-44e6-a287-eb8d19b4f9ce None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:41.706572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8775d591-0403-422c-9e3a-76e7872366df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:41.761594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fac7f-cbeb-41a4-8c6f-50dad25bbc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:41.763528 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fac7f-cbeb-41a4-8c6f-50dad25bbc1c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:41.764284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fac7f-cbeb-41a4-8c6f-50dad25bbc1c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:41.764765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fac7f-cbeb-41a4-8c6f-50dad25bbc1c None admin] PATH_INFO: `/v3/services/network` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:41.765644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1fac7f-cbeb-41a4-8c6f-50dad25bbc1c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:41.767272 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1fac7f-cbeb-41a4-8c6f-50dad25bbc1c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:41.778766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-0d1fac7f-cbeb-41a4-8c6f-50dad25bbc1c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:30:41.778766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:41.778766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:30:41.778766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:30:41.778766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:30:41.778766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:30:41.778766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:30:41.778766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:30:41.778766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:30:41.778766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:41.778766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:30:41.778766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:41.778766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:41.778766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:41.778766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:41.778766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:41.778766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:41.780117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:41.780117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:41.780117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:41.780117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:41.780117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:41.780117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:41.780117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:30:41.780117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:30:41.780117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:30:41.780117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:30:41.780117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:41.780117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:41.780117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:41.780117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:41.780117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:41.780117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:41.781114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:41.781114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:41.781114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:41.781114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:41.781114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:41.781114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:41.781114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:41.781114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:41.781114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:41.781114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:41.781114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:30:41.781114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:30:41.781114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:30:41.781114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:41.783275 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 361/716] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:41 2022] GET /identity/v3/services/network => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:41.784287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fac7f-cbeb-41a4-8c6f-50dad25bbc1c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:41.799934 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:41.864026 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 356/717] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:41 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:41.865472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775d591-0403-422c-9e3a-76e7872366df None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:41.990115 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fac7f-cbeb-41a4-8c6f-50dad25bbc1c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:42.075251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431844cf-3e62-446b-90a5-f964db21d626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:42.076304 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431844cf-3e62-446b-90a5-f964db21d626 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:42.076775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431844cf-3e62-446b-90a5-f964db21d626 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:42.077266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431844cf-3e62-446b-90a5-f964db21d626 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:42.099683 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775d591-0403-422c-9e3a-76e7872366df None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:42.104054 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431844cf-3e62-446b-90a5-f964db21d626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:42.107274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431844cf-3e62-446b-90a5-f964db21d626 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:42.125865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:42.150562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-431844cf-3e62-446b-90a5-f964db21d626 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:42.150562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:42.150562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:42.150562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:42.150562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:42.150562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:42.150562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:42.150562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:42.150562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:42.150562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:42.150562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:42.150562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:42.150562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:42.150562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:42.150562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:42.150562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:42.151658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:42.151658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:42.151658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:42.151658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:42.151658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:42.151658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:42.151658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:42.151658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:42.151658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:42.151658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:42.151658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:42.151658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:42.151658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:42.151658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:42.151658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:42.151658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:42.151658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:42.152544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:42.152544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:42.152544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:42.161166 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 362/718] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:42.162564 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431844cf-3e62-446b-90a5-f964db21d626 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:42.189169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec98347-ae70-42a2-9f3d-0ada96b76b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:42.189169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec98347-ae70-42a2-9f3d-0ada96b76b21 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:42.189169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec98347-ae70-42a2-9f3d-0ada96b76b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:42.189169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec98347-ae70-42a2-9f3d-0ada96b76b21 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:42.189783 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec98347-ae70-42a2-9f3d-0ada96b76b21 None admin] RBAC: Adding query filter params (name=network) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:42.189783 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec98347-ae70-42a2-9f3d-0ada96b76b21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:42.190233 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec98347-ae70-42a2-9f3d-0ada96b76b21 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:42.201180 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 357/719] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 26 05:30:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:42.229168 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec98347-ae70-42a2-9f3d-0ada96b76b21 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:42.434514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431844cf-3e62-446b-90a5-f964db21d626 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:42.436154 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:42.505989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048348cc-e76a-4cc8-81b4-6ad05adb7dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:42.507014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048348cc-e76a-4cc8-81b4-6ad05adb7dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:42.507513 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048348cc-e76a-4cc8-81b4-6ad05adb7dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:42.507979 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048348cc-e76a-4cc8-81b4-6ad05adb7dd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:42.508844 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048348cc-e76a-4cc8-81b4-6ad05adb7dd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:42.509400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048348cc-e76a-4cc8-81b4-6ad05adb7dd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:42.511073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048348cc-e76a-4cc8-81b4-6ad05adb7dd9 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:42.542208 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 363/720] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:42.545509 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048348cc-e76a-4cc8-81b4-6ad05adb7dd9 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:42.594793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec98347-ae70-42a2-9f3d-0ada96b76b21 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:42.674857 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893ea87-a1f8-4858-aa61-3b264324cdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:42.675936 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893ea87-a1f8-4858-aa61-3b264324cdee None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:42.676474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893ea87-a1f8-4858-aa61-3b264324cdee None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:42.676963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893ea87-a1f8-4858-aa61-3b264324cdee None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:42.677830 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893ea87-a1f8-4858-aa61-3b264324cdee None admin] RBAC: Adding query filter params (type=network) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:42.679563 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893ea87-a1f8-4858-aa61-3b264324cdee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:42.681234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893ea87-a1f8-4858-aa61-3b264324cdee None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:42.694380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:42.704910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 358/721] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 26 05:30:42 2022] GET /identity/v3/services?type=network => generated 124 bytes in 487 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:30:42.706105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893ea87-a1f8-4858-aa61-3b264324cdee None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:42.822627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048348cc-e76a-4cc8-81b4-6ad05adb7dd9 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:42.958036 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce391d-21c9-47a4-8f6d-0e61e0663eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:42.959778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce391d-21c9-47a4-8f6d-0e61e0663eda None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:42.960595 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce391d-21c9-47a4-8f6d-0e61e0663eda None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:42.961393 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce391d-21c9-47a4-8f6d-0e61e0663eda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:42.997374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce391d-21c9-47a4-8f6d-0e61e0663eda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:43.002028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-e3ce391d-21c9-47a4-8f6d-0e61e0663eda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:30:43.002028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:43.002028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:43.002028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:43.002028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:43.002028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:43.002028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:43.002028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:43.002028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:43.002028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:43.002028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:43.002028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:43.002028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:43.002028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:43.002028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:43.002028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:43.003271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:43.003271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:43.003271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:43.003271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:43.003271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:43.003271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:43.003271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:43.003271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:43.003271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:43.003271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:43.003271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:43.003271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:43.003271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:43.003271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:43.003271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:43.003271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:43.004468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:43.004468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:43.004468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:43.004468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:43.004468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:43.004468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:43.004468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:43.004468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:43.004468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:43.004468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:43.004468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:30:43.004468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:43.004468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 364/722] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:30:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 458 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:43.006783 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea4619-ab56-47bb-90ba-2c100790ac9c None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:43.006783 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea4619-ab56-47bb-90ba-2c100790ac9c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:43.006783 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea4619-ab56-47bb-90ba-2c100790ac9c None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:43.019977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 365/723] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:43.019977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea4619-ab56-47bb-90ba-2c100790ac9c None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:43.038990 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893ea87-a1f8-4858-aa61-3b264324cdee None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:43.041759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:43.089860 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ffd61-5b7c-4b5c-a12d-5094970dfab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:43.090516 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ffd61-5b7c-4b5c-a12d-5094970dfab4 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:43.090789 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ffd61-5b7c-4b5c-a12d-5094970dfab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:43.091057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ffd61-5b7c-4b5c-a12d-5094970dfab4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:43.113532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ffd61-5b7c-4b5c-a12d-5094970dfab4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:43.113532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ffd61-5b7c-4b5c-a12d-5094970dfab4 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:43.147464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-e90ffd61-5b7c-4b5c-a12d-5094970dfab4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:43.147464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:43.147464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:43.147464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:43.147464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:43.147464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:43.147464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:43.147464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:43.147464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:43.147464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:43.147464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:43.147464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:43.147464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:43.147464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:43.147464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:43.148474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:43.148474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:43.148474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:43.148474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:43.148474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:43.148474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:43.148474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:43.148474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:43.148474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:43.148474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:43.148474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:43.148474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:43.148474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:43.148474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:43.148474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:43.148474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:43.148474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:43.149390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:43.149390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:43.149390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:43.149390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:43.149390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:43.149390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:43.149390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:43.149390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:43.153710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 359/724] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 448 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:43.155383 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8595fe2-066c-4c48-8ffe-bd770ff851c9 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:43.157385 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8595fe2-066c-4c48-8ffe-bd770ff851c9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:43.159545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8595fe2-066c-4c48-8ffe-bd770ff851c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:43.203490 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f8595fe2-066c-4c48-8ffe-bd770ff851c9 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:43.206499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f8595fe2-066c-4c48-8ffe-bd770ff851c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:43.311273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea4619-ab56-47bb-90ba-2c100790ac9c None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:43.373988 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:43.408870 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041e78c-1446-413e-9091-fea4042e2e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:43.409916 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041e78c-1446-413e-9091-fea4042e2e3f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:43.410195 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041e78c-1446-413e-9091-fea4042e2e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:43.412374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041e78c-1446-413e-9091-fea4042e2e3f None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:43.413227 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041e78c-1446-413e-9091-fea4042e2e3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:43.413538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041e78c-1446-413e-9091-fea4042e2e3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:43.414936 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041e78c-1446-413e-9091-fea4042e2e3f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:43.416647 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 360/725] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:43 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:43.417742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8595fe2-066c-4c48-8ffe-bd770ff851c9 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:43.439567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 366/726] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:30:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:43.446233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041e78c-1446-413e-9091-fea4042e2e3f None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:43.610170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041e78c-1446-413e-9091-fea4042e2e3f None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:43.680269 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da713d5-cf46-44ca-9046-8e13714ab97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:43.680269 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da713d5-cf46-44ca-9046-8e13714ab97c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:43.680269 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da713d5-cf46-44ca-9046-8e13714ab97c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:43.680269 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da713d5-cf46-44ca-9046-8e13714ab97c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:43.701331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:43.717945 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da713d5-cf46-44ca-9046-8e13714ab97c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:43.725571 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da713d5-cf46-44ca-9046-8e13714ab97c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:43.741931 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8595fe2-066c-4c48-8ffe-bd770ff851c9 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:43.793454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-7da713d5-cf46-44ca-9046-8e13714ab97c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:43.793454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:43.793454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:43.793454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:43.793454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:43.793454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:43.793454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:43.793454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:43.793454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:43.793454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:43.793454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:43.793454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:43.793454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:43.793454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:43.793454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:43.793454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:43.794503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:43.794503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:43.794503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:43.794503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:43.794503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:43.794503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:43.794503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:43.794503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:43.794503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:43.794503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:43.794503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:43.794503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:43.794503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:43.794503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:43.794503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:43.794503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:43.794503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:43.795442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:43.795442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:43.795442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:43.802099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 367/727] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:43.817087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da713d5-cf46-44ca-9046-8e13714ab97c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:43.873248 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:43.874978 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338209c8-4023-4864-b47c-f22ba31dbef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:43.876392 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338209c8-4023-4864-b47c-f22ba31dbef2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:43.877075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338209c8-4023-4864-b47c-f22ba31dbef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:43.877656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338209c8-4023-4864-b47c-f22ba31dbef2 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:43.879940 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338209c8-4023-4864-b47c-f22ba31dbef2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:43.880158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338209c8-4023-4864-b47c-f22ba31dbef2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:43.882970 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338209c8-4023-4864-b47c-f22ba31dbef2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:43.912250 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 361/728] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 495 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:43.914549 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338209c8-4023-4864-b47c-f22ba31dbef2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:44.008783 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da713d5-cf46-44ca-9046-8e13714ab97c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:44.127141 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42e2e4-a8e1-4152-9d57-ec5f49d8fd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:44.128201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42e2e4-a8e1-4152-9d57-ec5f49d8fd63 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:44.128714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42e2e4-a8e1-4152-9d57-ec5f49d8fd63 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:44.129284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42e2e4-a8e1-4152-9d57-ec5f49d8fd63 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:44.130195 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42e2e4-a8e1-4152-9d57-ec5f49d8fd63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:44.130708 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42e2e4-a8e1-4152-9d57-ec5f49d8fd63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:44.133578 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42e2e4-a8e1-4152-9d57-ec5f49d8fd63 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:44.151357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 368/729] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:30:44.152754 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42e2e4-a8e1-4152-9d57-ec5f49d8fd63 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:44.285188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338209c8-4023-4864-b47c-f22ba31dbef2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:44.296827 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:44.311703 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42e2e4-a8e1-4152-9d57-ec5f49d8fd63 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:44.353234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adc62f-7c8e-46af-b4b3-7d37d28cd579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:44.353860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adc62f-7c8e-46af-b4b3-7d37d28cd579 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:44.354134 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adc62f-7c8e-46af-b4b3-7d37d28cd579 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:44.354399 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adc62f-7c8e-46af-b4b3-7d37d28cd579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:44.355210 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adc62f-7c8e-46af-b4b3-7d37d28cd579 None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=92556048cbd04c409c3f4b6fd645627f) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:44.355516 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adc62f-7c8e-46af-b4b3-7d37d28cd579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:44.357070 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adc62f-7c8e-46af-b4b3-7d37d28cd579 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:44.364865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 369/730] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:30:44 2022] GET /identity/v3/role_assignments?user.id=92556048cbd04c409c3f4b6fd645627f&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:44.364865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adc62f-7c8e-46af-b4b3-7d37d28cd579 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:44.385508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ab5c8-fe71-4505-9608-132456ba232a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:44.385508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ab5c8-fe71-4505-9608-132456ba232a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:44.385508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ab5c8-fe71-4505-9608-132456ba232a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:44.385508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ab5c8-fe71-4505-9608-132456ba232a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:44.408364 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ab5c8-fe71-4505-9608-132456ba232a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:44.416699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-a84ab5c8-fe71-4505-9608-132456ba232a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:44.416699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:44.416699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:44.416699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:44.416699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:44.416699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:44.416699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:44.416699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:44.416699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:44.416699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:44.416699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:44.416699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:44.416699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:44.416699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:44.416699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:44.416699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:44.418729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:44.418729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:44.418729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:44.418729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:44.418729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:44.418729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:44.418729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:44.418729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:44.418729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:44.418729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:44.418729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:44.418729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:44.418729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:44.418729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:44.418729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:44.418729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:44.420224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:44.420224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:44.420224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:44.420224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:44.420224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:44.420224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:44.420224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:44.420224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:44.420224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:44.420224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:44.420224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:44.420224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:44.420224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 362/731] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 506 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:44.422604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47938ea5-36a9-4bca-b826-f0891af54fbb None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:44.422604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47938ea5-36a9-4bca-b826-f0891af54fbb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:44.422604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47938ea5-36a9-4bca-b826-f0891af54fbb None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:44.429041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 363/732] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:44.437238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c64ac-b47d-416d-b0f5-de1bc5615946 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:44.437238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c64ac-b47d-416d-b0f5-de1bc5615946 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:44.437238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c64ac-b47d-416d-b0f5-de1bc5615946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:44.473174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-647c64ac-b47d-416d-b0f5-de1bc5615946 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:44.473174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-647c64ac-b47d-416d-b0f5-de1bc5615946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:44.497248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:44.609193 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adc62f-7c8e-46af-b4b3-7d37d28cd579 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:44.654281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 364/733] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:44.655503 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c64ac-b47d-416d-b0f5-de1bc5615946 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:44.691252 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59db35-1970-4ec7-9733-bbd4699671c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:44.692174 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59db35-1970-4ec7-9733-bbd4699671c9 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:44.692605 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59db35-1970-4ec7-9733-bbd4699671c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:44.693023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59db35-1970-4ec7-9733-bbd4699671c9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:44.704173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee59db35-1970-4ec7-9733-bbd4699671c9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:44.706657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee59db35-1970-4ec7-9733-bbd4699671c9 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:44.722159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-ee59db35-1970-4ec7-9733-bbd4699671c9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:30:44.722159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:44.722159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:44.722159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:44.722159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:44.722159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:44.722159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:44.722159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:44.722159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:44.722159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:44.722159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:44.722159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:44.722159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:44.722159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:44.722159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:44.722159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:44.723289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:44.724409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:44.724409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:44.724409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:44.724409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:44.724409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:44.724409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:44.724409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:44.724409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:44.724409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:30:44.724409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:44.727543 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 370/734] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:30:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:44.742822 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59db35-1970-4ec7-9733-bbd4699671c9 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:44.786042 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:44.822158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c64ac-b47d-416d-b0f5-de1bc5615946 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:44.871524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cb487-8d0e-4e3f-977a-6deeb3430e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:44.872470 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cb487-8d0e-4e3f-977a-6deeb3430e57 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:44.872903 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cb487-8d0e-4e3f-977a-6deeb3430e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:44.873396 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cb487-8d0e-4e3f-977a-6deeb3430e57 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:44.874278 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cb487-8d0e-4e3f-977a-6deeb3430e57 None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:44.874812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cb487-8d0e-4e3f-977a-6deeb3430e57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:44.876368 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cb487-8d0e-4e3f-977a-6deeb3430e57 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:44.886715 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 365/735] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:30:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:44.914962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cb487-8d0e-4e3f-977a-6deeb3430e57 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:44.924920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:45.063918 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59db35-1970-4ec7-9733-bbd4699671c9 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:45.170147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37b979-96da-4524-827f-7db4abfa4f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:45.170724 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37b979-96da-4524-827f-7db4abfa4f56 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:45.171012 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37b979-96da-4524-827f-7db4abfa4f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:45.171286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37b979-96da-4524-827f-7db4abfa4f56 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:45.172066 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37b979-96da-4524-827f-7db4abfa4f56 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:45.172371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37b979-96da-4524-827f-7db4abfa4f56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:45.178243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37b979-96da-4524-827f-7db4abfa4f56 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:45.243367 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cb487-8d0e-4e3f-977a-6deeb3430e57 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:45.243964 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 371/736] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Sep 26 05:30:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 502 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:45.245082 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37b979-96da-4524-827f-7db4abfa4f56 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:45.308901 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f6555-24c3-41b5-b704-399dc7c68b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:45.309972 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f6555-24c3-41b5-b704-399dc7c68b08 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:45.310488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f6555-24c3-41b5-b704-399dc7c68b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:45.310932 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f6555-24c3-41b5-b704-399dc7c68b08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:45.324421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f6555-24c3-41b5-b704-399dc7c68b08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:45.326260 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f6555-24c3-41b5-b704-399dc7c68b08 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:45.349209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-042f6555-24c3-41b5-b704-399dc7c68b08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:45.349209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:45.349209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:45.349209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:45.349209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:45.349209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:45.349209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:45.349209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:45.349209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:45.349209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:45.349209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:45.349209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:45.349209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:45.349209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:45.349209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:45.349209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:45.350320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:45.350320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:45.350320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:45.350320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:45.350320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:45.350320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:45.350320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:45.350320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:45.350320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:45.350320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:45.350320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:45.350320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:45.350320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:45.350320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:45.350320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:45.350320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:45.350320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:45.351347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:45.351347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:45.351347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:45.353496 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 366/737] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 440 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:45.355789 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcaaf5-123e-4d46-a353-47d0be053d97 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:45.356279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcaaf5-123e-4d46-a353-47d0be053d97 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:45.356751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcaaf5-123e-4d46-a353-47d0be053d97 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:45.363565 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:45.369533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 367/738] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:45 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:45.371185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcaaf5-123e-4d46-a353-47d0be053d97 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:45.425417 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37b979-96da-4524-827f-7db4abfa4f56 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:45.471760 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:45.496426 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e313865-f3ec-4008-b23a-48ffa95dae65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:45.497078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e313865-f3ec-4008-b23a-48ffa95dae65 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:45.497428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e313865-f3ec-4008-b23a-48ffa95dae65 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:45.497680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e313865-f3ec-4008-b23a-48ffa95dae65 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:45.498328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e313865-f3ec-4008-b23a-48ffa95dae65 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:45.499730 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e313865-f3ec-4008-b23a-48ffa95dae65 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:45.510420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-7e313865-f3ec-4008-b23a-48ffa95dae65 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:30:45.510420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:45.510420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:30:45.510420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:30:45.510420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:30:45.510420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:30:45.510420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:30:45.510420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:30:45.510420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:30:45.510420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:45.510420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:30:45.510420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:45.510420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:45.510420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:45.510420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:45.510420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:45.510420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:45.512374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:45.512374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:45.512374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:45.512374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:45.512374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:45.512374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:45.512374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:30:45.512374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:30:45.512374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:30:45.512374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:30:45.512374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:45.512374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:45.512374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:45.512374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:45.512374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:45.512374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:45.513968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:45.513968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:45.513968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:45.513968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:45.513968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:45.513968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:45.513968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:45.513968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:45.513968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:45.513968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:45.513968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:30:45.513968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:30:45.513968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:30:45.513968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:45.513968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 372/739] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Sep 26 05:30:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:45.515536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e313865-f3ec-4008-b23a-48ffa95dae65 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:45.543907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcaaf5-123e-4d46-a353-47d0be053d97 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:45.595153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3134ee6-a081-4d18-aab0-03253c897188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:45.596042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3134ee6-a081-4d18-aab0-03253c897188 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:45.596318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3134ee6-a081-4d18-aab0-03253c897188 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:45.596953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3134ee6-a081-4d18-aab0-03253c897188 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:45.676987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e313865-f3ec-4008-b23a-48ffa95dae65 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:45.700252 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3134ee6-a081-4d18-aab0-03253c897188 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:45.703858 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3134ee6-a081-4d18-aab0-03253c897188 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:45.739578 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37caa8-38b9-4d7a-a2ab-87f2afb427c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:45.741226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37caa8-38b9-4d7a-a2ab-87f2afb427c8 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:45.741953 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37caa8-38b9-4d7a-a2ab-87f2afb427c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:45.742716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37caa8-38b9-4d7a-a2ab-87f2afb427c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:45.744184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec37caa8-38b9-4d7a-a2ab-87f2afb427c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:45.745036 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec37caa8-38b9-4d7a-a2ab-87f2afb427c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:45.748136 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec37caa8-38b9-4d7a-a2ab-87f2afb427c8 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:45.754027 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:45.782025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 373/740] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:45.784992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ca266-de54-4b41-8315-a51eb1f6d9b4 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:45.786246 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ca266-de54-4b41-8315-a51eb1f6d9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:45.787706 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ca266-de54-4b41-8315-a51eb1f6d9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:45.797435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:45.831936 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d54ca266-de54-4b41-8315-a51eb1f6d9b4 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:45.834740 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d54ca266-de54-4b41-8315-a51eb1f6d9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:45.884518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 368/741] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:30:45 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 05:30:45.884518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3134ee6-a081-4d18-aab0-03253c897188 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:45.964359 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 374/742] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 05:30:45.965766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ca266-de54-4b41-8315-a51eb1f6d9b4 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:46.125521 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3134ee6-a081-4d18-aab0-03253c897188 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:46.154162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ca266-de54-4b41-8315-a51eb1f6d9b4 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:46.202759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:46.211537 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9845371-2560-4b1e-942c-f2e1d12cce8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:46.212692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9845371-2560-4b1e-942c-f2e1d12cce8d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:46.218366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9845371-2560-4b1e-942c-f2e1d12cce8d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:46.220245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9845371-2560-4b1e-942c-f2e1d12cce8d None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:46.222450 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9845371-2560-4b1e-942c-f2e1d12cce8d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:46.224340 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9845371-2560-4b1e-942c-f2e1d12cce8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:46.231502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9845371-2560-4b1e-942c-f2e1d12cce8d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:46.250129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9541a5-f56d-4cc6-bbad-bdf66597196e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:46.250129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9541a5-f56d-4cc6-bbad-bdf66597196e None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:46.250129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9541a5-f56d-4cc6-bbad-bdf66597196e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:46.250129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9541a5-f56d-4cc6-bbad-bdf66597196e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:46.253185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 369/743] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Sep 26 05:30:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:46.253185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9845371-2560-4b1e-942c-f2e1d12cce8d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:46.279318 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9541a5-f56d-4cc6-bbad-bdf66597196e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:46.279318 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9541a5-f56d-4cc6-bbad-bdf66597196e None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:46.327448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:46.327778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-7d9541a5-f56d-4cc6-bbad-bdf66597196e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:30:46.327778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:46.327778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:46.327778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:46.327778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:46.327778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:46.327778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:46.327778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:46.327778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:46.327778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:46.327778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:46.327778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:46.327778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:46.327778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:46.327778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:46.327778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:46.328675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:46.329672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:46.329672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:46.329672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:46.329672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:46.329672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:46.329672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:46.329672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:46.329672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:46.329672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:30:46.329672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:46.337043 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 375/744] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 26 05:30:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:46.337043 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9541a5-f56d-4cc6-bbad-bdf66597196e None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:46.543008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9541a5-f56d-4cc6-bbad-bdf66597196e None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:46.546568 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9845371-2560-4b1e-942c-f2e1d12cce8d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:46.595597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdca527-61ca-4c02-83d2-48c5cb792445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:46.597547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdca527-61ca-4c02-83d2-48c5cb792445 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:46.598093 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdca527-61ca-4c02-83d2-48c5cb792445 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:46.598854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdca527-61ca-4c02-83d2-48c5cb792445 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:46.606156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32867e2-1ed5-421b-9338-c055b551787a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:46.607186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32867e2-1ed5-421b-9338-c055b551787a None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:46.609267 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32867e2-1ed5-421b-9338-c055b551787a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:46.609267 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32867e2-1ed5-421b-9338-c055b551787a None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:46.609267 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32867e2-1ed5-421b-9338-c055b551787a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:46.610392 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32867e2-1ed5-421b-9338-c055b551787a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:46.615663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32867e2-1ed5-421b-9338-c055b551787a None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:46.619169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdca527-61ca-4c02-83d2-48c5cb792445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:46.622034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdca527-61ca-4c02-83d2-48c5cb792445 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:46.627455 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 376/745] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Sep 26 05:30:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:46.627455 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32867e2-1ed5-421b-9338-c055b551787a None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:46.660077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-4cdca527-61ca-4c02-83d2-48c5cb792445 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:46.660077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:46.660077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:46.660077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:46.660077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:46.660077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:46.660077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:46.660077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:46.660077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:46.660077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:46.660077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:46.660077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:46.660077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:46.660077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:46.660077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:46.660077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:46.661125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:46.661125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:46.661125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:46.661125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:46.661125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:46.661125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:46.661125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:46.661125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:46.661125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:46.661125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:46.661125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:46.661125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:46.661125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:46.661125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:46.661125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:46.661125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:46.661125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:46.662104 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:46.662104 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:46.662104 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:46.662367 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 370/746] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:46.663880 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdca527-61ca-4c02-83d2-48c5cb792445 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:46.725063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:46.813384 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32867e2-1ed5-421b-9338-c055b551787a None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:46.819646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:46.866999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdca527-61ca-4c02-83d2-48c5cb792445 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:46.882227 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197ba71-3f18-45f3-87ed-10cfc57ded74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:46.883177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197ba71-3f18-45f3-87ed-10cfc57ded74 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:46.883649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197ba71-3f18-45f3-87ed-10cfc57ded74 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:46.884119 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197ba71-3f18-45f3-87ed-10cfc57ded74 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:46.885334 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197ba71-3f18-45f3-87ed-10cfc57ded74 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:46.885790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197ba71-3f18-45f3-87ed-10cfc57ded74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:46.888414 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197ba71-3f18-45f3-87ed-10cfc57ded74 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:46.908463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5026c-4cf6-4aca-9d62-e9bcbc2ad9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:46.909800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5026c-4cf6-4aca-9d62-e9bcbc2ad9f2 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:46.910291 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5026c-4cf6-4aca-9d62-e9bcbc2ad9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:46.910764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5026c-4cf6-4aca-9d62-e9bcbc2ad9f2 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:46.911500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5026c-4cf6-4aca-9d62-e9bcbc2ad9f2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:46.913004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5026c-4cf6-4aca-9d62-e9bcbc2ad9f2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:46.921189 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 377/747] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Sep 26 05:30:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:46.921727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197ba71-3f18-45f3-87ed-10cfc57ded74 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:46.942521 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 371/748] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 26 05:30:46 2022] POST /identity/v3/services => generated 240 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:46.943031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5026c-4cf6-4aca-9d62-e9bcbc2ad9f2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:47.098458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197ba71-3f18-45f3-87ed-10cfc57ded74 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:47.113830 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:47.140441 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5026c-4cf6-4aca-9d62-e9bcbc2ad9f2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:47.148405 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2ef73-8081-470b-b414-9b3074fce494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:47.149061 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2ef73-8081-470b-b414-9b3074fce494 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:47.149444 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2ef73-8081-470b-b414-9b3074fce494 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:47.149684 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2ef73-8081-470b-b414-9b3074fce494 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:47.150405 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd2ef73-8081-470b-b414-9b3074fce494 None admin] RBAC: Adding query filter params (interface=public, service_id=087c9991d9f5496f955ca34290661813, region_id=RegionOne) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:47.150724 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd2ef73-8081-470b-b414-9b3074fce494 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:47.152237 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd2ef73-8081-470b-b414-9b3074fce494 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:47.169821 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 378/749] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Sep 26 05:30:46 2022] GET /identity/v3/endpoints?service_id=087c9991d9f5496f955ca34290661813&interface=public®ion_id=RegionOne => generated 194 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:47.169821 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaba8a1-2f64-4b42-86f7-e7d625fd447e None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:47.169821 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaba8a1-2f64-4b42-86f7-e7d625fd447e None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:47.169821 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaba8a1-2f64-4b42-86f7-e7d625fd447e None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:47.171252 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 379/750] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:47.172268 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaba8a1-2f64-4b42-86f7-e7d625fd447e None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:47.206799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:47.221045 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45727d1b-6cc3-4959-9a7a-1e7040ed6e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:47.222081 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45727d1b-6cc3-4959-9a7a-1e7040ed6e2c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:47.222359 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45727d1b-6cc3-4959-9a7a-1e7040ed6e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:47.222633 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45727d1b-6cc3-4959-9a7a-1e7040ed6e2c None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:47.223313 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45727d1b-6cc3-4959-9a7a-1e7040ed6e2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:47.224290 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45727d1b-6cc3-4959-9a7a-1e7040ed6e2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:47.225267 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45727d1b-6cc3-4959-9a7a-1e7040ed6e2c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:47.246083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 372/751] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:47.247321 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45727d1b-6cc3-4959-9a7a-1e7040ed6e2c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:47.369441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaba8a1-2f64-4b42-86f7-e7d625fd447e None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:47.403670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45727d1b-6cc3-4959-9a7a-1e7040ed6e2c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:47.414918 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be9867-9ba7-4141-8da6-957db923f477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:47.415592 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be9867-9ba7-4141-8da6-957db923f477 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:47.415864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be9867-9ba7-4141-8da6-957db923f477 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:47.416133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be9867-9ba7-4141-8da6-957db923f477 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:47.473734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:47.475223 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e602a-b7b3-490f-98ce-552cc5c73995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:47.476603 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e602a-b7b3-490f-98ce-552cc5c73995 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:47.477427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e602a-b7b3-490f-98ce-552cc5c73995 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:47.478189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e602a-b7b3-490f-98ce-552cc5c73995 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:47.479816 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e602a-b7b3-490f-98ce-552cc5c73995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:47.482451 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e602a-b7b3-490f-98ce-552cc5c73995 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:47.505548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 373/752] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 26 05:30:47 2022] GET /identity/v3/services => generated 772 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:47.506960 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a16534-aa6c-4c7c-9dc3-6f44ef828ed8 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:47.507251 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a16534-aa6c-4c7c-9dc3-6f44ef828ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:47.507636 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a16534-aa6c-4c7c-9dc3-6f44ef828ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:47.525485 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be9867-9ba7-4141-8da6-957db923f477 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:47.527459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be9867-9ba7-4141-8da6-957db923f477 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:47.574437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f1a16534-aa6c-4c7c-9dc3-6f44ef828ed8 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:47.575682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f1a16534-aa6c-4c7c-9dc3-6f44ef828ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:47.668106 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 380/753] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:30:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:30:47.670426 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be9867-9ba7-4141-8da6-957db923f477 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:47.704377 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:47.789688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 374/754] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:30:47.791213 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a16534-aa6c-4c7c-9dc3-6f44ef828ed8 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:47.901377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:47.949280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be9867-9ba7-4141-8da6-957db923f477 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:47.995112 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ef9dd-cf26-4044-b8ca-9759bc08fc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:47.996301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ef9dd-cf26-4044-b8ca-9759bc08fc37 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:47.996819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ef9dd-cf26-4044-b8ca-9759bc08fc37 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:47.997349 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ef9dd-cf26-4044-b8ca-9759bc08fc37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:48.006688 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ef9dd-cf26-4044-b8ca-9759bc08fc37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:48.008542 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ef9dd-cf26-4044-b8ca-9759bc08fc37 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:48.022597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-cb1ef9dd-cf26-4044-b8ca-9759bc08fc37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:48.022597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:48.022597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:48.022597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:48.022597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:48.022597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:48.022597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:48.022597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:48.022597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:48.022597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:48.022597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:48.022597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:48.022597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:48.022597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:48.022597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:48.023581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:48.023581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:48.023581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:48.023581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:48.023581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:48.023581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:48.023581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:48.023581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:48.023581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:48.023581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:48.023581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:48.023581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:48.023581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:48.023581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:48.023581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:48.023581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:48.023581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:48.024531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:48.024531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:48.024531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:48.024531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:48.024531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:48.024531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:48.024531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:48.024531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:48.028266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 381/755] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:48.037584 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ef9dd-cf26-4044-b8ca-9759bc08fc37 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:48.226758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a16534-aa6c-4c7c-9dc3-6f44ef828ed8 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:48.428353 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e0d7a-15e4-4921-9681-1d6c44300e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:48.433170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e0d7a-15e4-4921-9681-1d6c44300e98 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:48.433170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e0d7a-15e4-4921-9681-1d6c44300e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:48.433170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e0d7a-15e4-4921-9681-1d6c44300e98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:48.444203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ef9dd-cf26-4044-b8ca-9759bc08fc37 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:48.477276 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e0d7a-15e4-4921-9681-1d6c44300e98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:48.485163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e0d7a-15e4-4921-9681-1d6c44300e98 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:48.524065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-130e0d7a-15e4-4921-9681-1d6c44300e98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:48.524065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:48.524065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:48.524065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:48.524065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:48.524065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:48.524065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:48.524065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:48.524065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:48.524065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:48.524065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:48.524065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:48.524065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:48.524065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:48.524065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:48.524065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:48.525912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:48.525912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:48.525912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:48.525912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:48.525912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:48.525912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:48.525912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:48.525912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:48.525912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:48.525912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:48.525912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:48.525912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:48.525912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:48.525912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:48.525912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:48.525912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:48.525912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:48.527683 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:48.527683 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:48.527683 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:48.532130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 375/756] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 738 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:48.532130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e0d7a-15e4-4921-9681-1d6c44300e98 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:48.539915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:48.577172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbaa27-c9b5-447f-ac1e-d3f7afe68aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:48.577172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbaa27-c9b5-447f-ac1e-d3f7afe68aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:48.577172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbaa27-c9b5-447f-ac1e-d3f7afe68aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:48.577172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbaa27-c9b5-447f-ac1e-d3f7afe68aa1 None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:48.578284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbaa27-c9b5-447f-ac1e-d3f7afe68aa1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:48.578512 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbaa27-c9b5-447f-ac1e-d3f7afe68aa1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:48.589166 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbaa27-c9b5-447f-ac1e-d3f7afe68aa1 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:48.599680 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:48.632857 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 382/757] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 596 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:48.639058 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbaa27-c9b5-447f-ac1e-d3f7afe68aa1 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:48.681825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e0d7a-15e4-4921-9681-1d6c44300e98 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:48.720348 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec0c56-a53e-490c-bf63-66e7190660a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:48.721772 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec0c56-a53e-490c-bf63-66e7190660a9 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:48.722276 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec0c56-a53e-490c-bf63-66e7190660a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:48.722757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec0c56-a53e-490c-bf63-66e7190660a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:48.735461 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec0c56-a53e-490c-bf63-66e7190660a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:48.741578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-98ec0c56-a53e-490c-bf63-66e7190660a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:30:48.741578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:48.741578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:48.741578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:48.741578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:48.741578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:48.741578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:48.741578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:48.741578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:48.741578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:48.741578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:48.741578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:48.741578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:48.741578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:48.741578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:48.741578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:48.742596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:48.742596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:48.742596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:48.742596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:48.742596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:48.742596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:48.742596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:48.742596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:48.742596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:48.742596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:48.742596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:48.742596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:48.742596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:48.742596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:48.742596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:48.742596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:48.743511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:48.743511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:48.743511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:48.743511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:48.743511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:48.743511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:48.743511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:48.743511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:48.743511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:48.743511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:48.743511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:30:48.743511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:48.746472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 376/758] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:30:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:48.747711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec0c56-a53e-490c-bf63-66e7190660a9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:48.924687 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbaa27-c9b5-447f-ac1e-d3f7afe68aa1 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:48.979289 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:49.000969 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec0c56-a53e-490c-bf63-66e7190660a9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:49.059699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd962fe-8725-46da-aa4b-d54def990381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.062879 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd962fe-8725-46da-aa4b-d54def990381 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.062879 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd962fe-8725-46da-aa4b-d54def990381 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.062879 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd962fe-8725-46da-aa4b-d54def990381 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.063606 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd962fe-8725-46da-aa4b-d54def990381 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:49.064149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd962fe-8725-46da-aa4b-d54def990381 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.067793 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd962fe-8725-46da-aa4b-d54def990381 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.109730 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 383/759] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 476 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:49.113831 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd962fe-8725-46da-aa4b-d54def990381 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.130376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475fa4b-a16e-4385-afe2-ee7dae7dada5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.132059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475fa4b-a16e-4385-afe2-ee7dae7dada5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.132879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475fa4b-a16e-4385-afe2-ee7dae7dada5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.133612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475fa4b-a16e-4385-afe2-ee7dae7dada5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.135376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475fa4b-a16e-4385-afe2-ee7dae7dada5 None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=00f258aba84d4dd59ebe8de973ac65c6) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:49.136299 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475fa4b-a16e-4385-afe2-ee7dae7dada5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.145040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3475fa4b-a16e-4385-afe2-ee7dae7dada5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.164977 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 377/760] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:30:48 2022] GET /identity/v3/role_assignments?user.id=00f258aba84d4dd59ebe8de973ac65c6&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 259 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:49.167149 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475fa4b-a16e-4385-afe2-ee7dae7dada5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.402022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:49.578862 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475fa4b-a16e-4385-afe2-ee7dae7dada5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:49.671706 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd962fe-8725-46da-aa4b-d54def990381 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:49.722938 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60157a3a-35cf-4315-a806-3539465890ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.724159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60157a3a-35cf-4315-a806-3539465890ca None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.724700 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60157a3a-35cf-4315-a806-3539465890ca None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.725292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60157a3a-35cf-4315-a806-3539465890ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.753374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60157a3a-35cf-4315-a806-3539465890ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.756311 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60157a3a-35cf-4315-a806-3539465890ca None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.797898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-60157a3a-35cf-4315-a806-3539465890ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:49.797898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:49.797898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:49.797898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:49.797898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:49.797898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:49.797898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:49.797898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:49.797898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:49.797898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:49.797898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:49.797898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:49.797898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:49.797898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:49.797898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:49.799894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:49.799894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:49.799894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:49.799894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:49.799894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:49.799894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:49.799894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:49.799894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:49.799894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:49.799894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:49.799894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:49.799894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:49.799894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:49.799894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:49.799894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:49.799894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:49.799894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:49.801962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:49.801962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:49.801962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:49.801962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:49.801962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:49.801962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:49.801962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:49.801962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:49.811782 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a49b7-2704-4b61-828a-83195bc7ab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:49.812960 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a49b7-2704-4b61-828a-83195bc7ab87 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:49.813636 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a49b7-2704-4b61-828a-83195bc7ab87 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:49.814143 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a49b7-2704-4b61-828a-83195bc7ab87 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:49.815089 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 378/761] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 649 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:49.816541 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a49b7-2704-4b61-828a-83195bc7ab87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:49.817092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a49b7-2704-4b61-828a-83195bc7ab87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:49.821202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60157a3a-35cf-4315-a806-3539465890ca None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.824787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a49b7-2704-4b61-828a-83195bc7ab87 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:49.860953 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 384/762] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:30:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 750 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:49.877168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a49b7-2704-4b61-828a-83195bc7ab87 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:49.918185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:50.341111 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a49b7-2704-4b61-828a-83195bc7ab87 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:50.416319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60157a3a-35cf-4315-a806-3539465890ca None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:50.623283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5160e-3879-48bc-8d1e-1a38395990b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:50.624318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5160e-3879-48bc-8d1e-1a38395990b7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:50.624318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5160e-3879-48bc-8d1e-1a38395990b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:50.624578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5160e-3879-48bc-8d1e-1a38395990b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:50.629166 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5160e-3879-48bc-8d1e-1a38395990b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:50.629166 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5160e-3879-48bc-8d1e-1a38395990b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:50.629166 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5160e-3879-48bc-8d1e-1a38395990b7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:50.686290 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 379/763] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 865 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:50.689172 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c5ec2-f5bf-4565-a8c2-41ff78467afe None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:50.689172 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c5ec2-f5bf-4565-a8c2-41ff78467afe None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:50.689172 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c5ec2-f5bf-4565-a8c2-41ff78467afe None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:50.701964 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 380/764] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:50 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:50.705163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c5ec2-f5bf-4565-a8c2-41ff78467afe None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:50.828864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:50.899747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f6fa60-5cc7-42a6-b32e-dbae400e87b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:50.901337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6fa60-5cc7-42a6-b32e-dbae400e87b4 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:50.902203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6fa60-5cc7-42a6-b32e-dbae400e87b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:50.903373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6fa60-5cc7-42a6-b32e-dbae400e87b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:50.925968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6fa60-5cc7-42a6-b32e-dbae400e87b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:50.928543 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:50.929716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6fa60-5cc7-42a6-b32e-dbae400e87b4 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:50.958638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c5ec2-f5bf-4565-a8c2-41ff78467afe None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:50.958914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-a6f6fa60-5cc7-42a6-b32e-dbae400e87b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:50.958914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:50.958914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:50.958914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:50.958914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:50.958914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:50.958914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:50.958914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:50.958914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:50.958914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:50.958914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:50.958914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:50.958914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:50.958914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:50.958914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:50.958914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:50.960640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:50.960640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:50.960640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:50.960640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:50.960640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:50.960640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:50.960640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:50.960640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:50.960640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:50.960640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:50.960640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:50.960640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:50.960640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:50.960640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:50.960640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:50.960640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:50.960640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:50.962396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:50.962396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:50.962396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:50.962396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 385/765] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 1092 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:50.967406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153033d3-1e89-4b90-bb87-0277323c6470 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:50.967406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153033d3-1e89-4b90-bb87-0277323c6470 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:50.967406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153033d3-1e89-4b90-bb87-0277323c6470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:50.993769 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-153033d3-1e89-4b90-bb87-0277323c6470 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:50.995792 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-153033d3-1e89-4b90-bb87-0277323c6470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:51.007365 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ebc55-0410-45c2-91c5-e2863d524225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:51.008439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ebc55-0410-45c2-91c5-e2863d524225 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:51.008876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ebc55-0410-45c2-91c5-e2863d524225 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:51.009357 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ebc55-0410-45c2-91c5-e2863d524225 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:51.010302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ebc55-0410-45c2-91c5-e2863d524225 None admin] RBAC: Adding query filter params (role.id=881210e567974dc0aa3043393c00a838, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=0216bf67389e4627bf953b7dcc97a88c) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:51.010767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ebc55-0410-45c2-91c5-e2863d524225 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:51.013326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ebc55-0410-45c2-91c5-e2863d524225 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:51.027892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 381/766] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:30:50 2022] GET /identity/v3/role_assignments?user.id=0216bf67389e4627bf953b7dcc97a88c&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=881210e567974dc0aa3043393c00a838 => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:51.027892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab56c2-7f71-413d-a502-25eca874df71 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:51.027892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab56c2-7f71-413d-a502-25eca874df71 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:51.027892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab56c2-7f71-413d-a502-25eca874df71 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:51.027892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 382/767] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:51.027892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab56c2-7f71-413d-a502-25eca874df71 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:51.157610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 386/768] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:30:51.162849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d53f6-65ed-43c8-9093-30e45ed2b1f8 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:51.162849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d53f6-65ed-43c8-9093-30e45ed2b1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:51.163450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d53f6-65ed-43c8-9093-30e45ed2b1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:51.239963 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-437d53f6-65ed-43c8-9093-30e45ed2b1f8 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:51.241221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-437d53f6-65ed-43c8-9093-30e45ed2b1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:51.285012 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab56c2-7f71-413d-a502-25eca874df71 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:51.339355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee0011-2484-44a6-b8ba-5ba3647df318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:51.339988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee0011-2484-44a6-b8ba-5ba3647df318 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:51.340270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee0011-2484-44a6-b8ba-5ba3647df318 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:51.340522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee0011-2484-44a6-b8ba-5ba3647df318 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:51.347019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee0011-2484-44a6-b8ba-5ba3647df318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:51.347466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee0011-2484-44a6-b8ba-5ba3647df318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:51.348964 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee0011-2484-44a6-b8ba-5ba3647df318 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:51.362744 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 387/769] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:30:51.375630 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 383/770] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:51.381655 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d53f6-65ed-43c8-9093-30e45ed2b1f8 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:51.428582 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:51.513012 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d53f6-65ed-43c8-9093-30e45ed2b1f8 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:51.554643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a676d3-1b7c-4ef6-8e6a-2f9965f92e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:51.555369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a676d3-1b7c-4ef6-8e6a-2f9965f92e18 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:51.555648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a676d3-1b7c-4ef6-8e6a-2f9965f92e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:51.555923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a676d3-1b7c-4ef6-8e6a-2f9965f92e18 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:51.565024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a676d3-1b7c-4ef6-8e6a-2f9965f92e18 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:51.566571 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a676d3-1b7c-4ef6-8e6a-2f9965f92e18 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:51.580129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-e3a676d3-1b7c-4ef6-8e6a-2f9965f92e18 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:30:51.580129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:51.580129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:51.580129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:51.580129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:51.580129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:51.580129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:51.580129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:51.580129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:51.580129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:51.580129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:51.580129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:51.580129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:51.580129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:51.580129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:51.580129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:51.582251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:51.584271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:51.584271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:51.584271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:51.584271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:51.584271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:51.584271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:51.584271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:51.584271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:51.584271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:30:51.584271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:51.584271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 388/771] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 26 05:30:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:51.587362 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee0011-2484-44a6-b8ba-5ba3647df318 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:51.590489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:51.691333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a676d3-1b7c-4ef6-8e6a-2f9965f92e18 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:51.788740 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee0011-2484-44a6-b8ba-5ba3647df318 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:51.828744 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc27ec1-cb56-4cb3-8c09-4c87ef96a4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:51.829571 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc27ec1-cb56-4cb3-8c09-4c87ef96a4df None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:51.829945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc27ec1-cb56-4cb3-8c09-4c87ef96a4df None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:51.830309 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc27ec1-cb56-4cb3-8c09-4c87ef96a4df None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:51.831171 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc27ec1-cb56-4cb3-8c09-4c87ef96a4df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:51.831620 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc27ec1-cb56-4cb3-8c09-4c87ef96a4df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:51.833250 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc27ec1-cb56-4cb3-8c09-4c87ef96a4df None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:51.853845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 384/772] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 26 05:30:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:51.857936 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:51.858437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc27ec1-cb56-4cb3-8c09-4c87ef96a4df None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:51.897586 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a676d3-1b7c-4ef6-8e6a-2f9965f92e18 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:51.935682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:52.003989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc27ec1-cb56-4cb3-8c09-4c87ef96a4df None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:52.006336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f4a17-2f10-4f87-adc4-d79395a38ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:52.007849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f4a17-2f10-4f87-adc4-d79395a38ace None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:52.008672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f4a17-2f10-4f87-adc4-d79395a38ace None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:52.009644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f4a17-2f10-4f87-adc4-d79395a38ace None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:52.011238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f4a17-2f10-4f87-adc4-d79395a38ace None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:52.014697 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f4a17-2f10-4f87-adc4-d79395a38ace None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:52.046868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-0a7f4a17-2f10-4f87-adc4-d79395a38ace None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:30:52.046868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:52.046868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:30:52.046868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:30:52.046868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:30:52.046868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:30:52.046868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:30:52.046868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:30:52.046868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:30:52.046868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:52.046868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:30:52.046868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:52.046868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:52.046868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:52.046868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:52.046868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:52.046868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:52.048814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:52.048814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:52.048814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:52.048814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:52.048814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:52.048814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:52.048814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:30:52.048814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:30:52.048814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:30:52.048814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:30:52.048814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:52.048814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:52.048814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:52.048814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:52.048814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:52.048814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:52.050577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:52.050577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:52.050577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:52.050577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:52.050577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:52.050577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:52.050577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:52.050577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:52.050577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:52.050577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:52.050577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:30:52.050577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:30:52.050577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 05:30:52.050577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:52.050577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 389/773] 10.222.0.57 () {56 vars in 1129 bytes} [Mon Sep 26 05:30:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:52.052141 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f4a17-2f10-4f87-adc4-d79395a38ace None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:52.072799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc249cf-12a3-46be-8e21-ed51c48cda14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:52.072799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc249cf-12a3-46be-8e21-ed51c48cda14 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:52.072799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc249cf-12a3-46be-8e21-ed51c48cda14 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:52.072799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc249cf-12a3-46be-8e21-ed51c48cda14 None admin] PATH_INFO: `/v3/services/network` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:52.073844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc249cf-12a3-46be-8e21-ed51c48cda14 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:52.073844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc249cf-12a3-46be-8e21-ed51c48cda14 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:52.122916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-cfc249cf-12a3-46be-8e21-ed51c48cda14 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:30:52.122916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:52.122916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:30:52.122916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:30:52.122916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:30:52.122916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:30:52.122916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:30:52.122916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:30:52.122916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:30:52.122916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:52.122916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:30:52.122916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:52.122916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:52.122916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:52.122916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:52.122916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:52.122916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:52.124897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:52.124897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:52.124897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:52.124897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:52.124897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:52.124897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:52.124897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:30:52.124897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:30:52.124897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:30:52.124897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:30:52.124897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:52.124897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:52.124897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:52.124897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:52.124897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:52.124897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:52.126672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:52.126672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:52.126672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:52.126672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:52.126672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:52.126672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:52.126672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:52.126672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:52.126672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:52.126672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:52.126672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:30:52.126672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:30:52.126672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:30:52.126672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:52.136149 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 385/774] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:51 2022] GET /identity/v3/services/network => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:52.138173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc249cf-12a3-46be-8e21-ed51c48cda14 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:52.206866 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f4a17-2f10-4f87-adc4-d79395a38ace None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:52.269111 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5886f-d083-471e-ac31-6b6750a348e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:52.269696 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5886f-d083-471e-ac31-6b6750a348e0 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:52.269976 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5886f-d083-471e-ac31-6b6750a348e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:52.270283 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5886f-d083-471e-ac31-6b6750a348e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:52.291604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5886f-d083-471e-ac31-6b6750a348e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:52.293172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5886f-d083-471e-ac31-6b6750a348e0 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:52.338660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-f2f5886f-d083-471e-ac31-6b6750a348e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:52.338660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:52.338660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:52.338660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:52.338660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:52.338660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:52.338660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:52.338660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:52.338660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:52.338660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:52.338660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:52.338660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:52.338660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:52.338660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:52.338660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:52.338660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:52.340602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:52.340602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:52.340602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:52.340602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:52.340602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:52.340602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:52.340602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:52.340602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:52.340602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:52.340602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:52.340602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:52.340602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:52.340602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:52.340602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:52.340602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:52.340602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:52.340602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:52.342386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:52.342386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:52.342386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:52.346313 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 390/775] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:52.348772 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5886f-d083-471e-ac31-6b6750a348e0 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:52.365158 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:52.432222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc249cf-12a3-46be-8e21-ed51c48cda14 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:52.472223 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:52.490162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bed0b-3ae5-42d1-91f8-3261ecf3f7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:52.491048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bed0b-3ae5-42d1-91f8-3261ecf3f7de None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:52.491467 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bed0b-3ae5-42d1-91f8-3261ecf3f7de None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:52.492048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bed0b-3ae5-42d1-91f8-3261ecf3f7de None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:52.492857 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4bed0b-3ae5-42d1-91f8-3261ecf3f7de None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:52.493718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4bed0b-3ae5-42d1-91f8-3261ecf3f7de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:52.495775 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4bed0b-3ae5-42d1-91f8-3261ecf3f7de None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:52.502121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 386/776] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Sep 26 05:30:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:52.503617 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bed0b-3ae5-42d1-91f8-3261ecf3f7de None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:52.555133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5886f-d083-471e-ac31-6b6750a348e0 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:52.596290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c02e2-4eaa-406c-9eab-1232e8aaef15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:52.597371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c02e2-4eaa-406c-9eab-1232e8aaef15 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:52.598118 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c02e2-4eaa-406c-9eab-1232e8aaef15 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:52.598439 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c02e2-4eaa-406c-9eab-1232e8aaef15 None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:52.599073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c02e2-4eaa-406c-9eab-1232e8aaef15 None admin] RBAC: Adding query filter params (name=network) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:52.599773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c02e2-4eaa-406c-9eab-1232e8aaef15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:52.601516 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c02e2-4eaa-406c-9eab-1232e8aaef15 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:52.609565 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 391/777] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 26 05:30:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:52.611630 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c6e24-cf6c-425d-8548-636c9f2e33c9 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:52.612082 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c6e24-cf6c-425d-8548-636c9f2e33c9 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:52.612823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c6e24-cf6c-425d-8548-636c9f2e33c9 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:52.614876 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 392/778] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:52.616136 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c6e24-cf6c-425d-8548-636c9f2e33c9 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:52.628592 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bed0b-3ae5-42d1-91f8-3261ecf3f7de None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:52.679760 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0a704-2298-45cb-aaa8-cc7857f99db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:52.680634 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0a704-2298-45cb-aaa8-cc7857f99db2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:52.681022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0a704-2298-45cb-aaa8-cc7857f99db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:52.681319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0a704-2298-45cb-aaa8-cc7857f99db2 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:52.681981 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0a704-2298-45cb-aaa8-cc7857f99db2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:52.682747 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0a704-2298-45cb-aaa8-cc7857f99db2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:52.683731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0a704-2298-45cb-aaa8-cc7857f99db2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:52.698072 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 387/779] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:52.703401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0a704-2298-45cb-aaa8-cc7857f99db2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:52.730052 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:52.767683 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:52.790522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c6e24-cf6c-425d-8548-636c9f2e33c9 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:52.831040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0a704-2298-45cb-aaa8-cc7857f99db2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:52.835851 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c67d1-b1f1-4415-be79-6bbf34d58f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:52.836706 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c67d1-b1f1-4415-be79-6bbf34d58f64 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:52.837181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c67d1-b1f1-4415-be79-6bbf34d58f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:52.837597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c67d1-b1f1-4415-be79-6bbf34d58f64 None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:52.838387 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c67d1-b1f1-4415-be79-6bbf34d58f64 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:52.838833 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c67d1-b1f1-4415-be79-6bbf34d58f64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:52.840548 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c67d1-b1f1-4415-be79-6bbf34d58f64 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:52.848717 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 393/780] 10.222.0.57 () {56 vars in 1124 bytes} [Mon Sep 26 05:30:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:52.850643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba42691-f6e4-4e90-97a2-6a9353eb5a4f None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:52.851079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba42691-f6e4-4e90-97a2-6a9353eb5a4f None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:52.851352 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba42691-f6e4-4e90-97a2-6a9353eb5a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:52.875293 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc537fdf-a615-4b5e-85c3-cfb48b2d2de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:52.876005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc537fdf-a615-4b5e-85c3-cfb48b2d2de5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:52.876348 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc537fdf-a615-4b5e-85c3-cfb48b2d2de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:52.876688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc537fdf-a615-4b5e-85c3-cfb48b2d2de5 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:52.877430 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc537fdf-a615-4b5e-85c3-cfb48b2d2de5 None admin] RBAC: Adding query filter params (type=network) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:52.877792 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc537fdf-a615-4b5e-85c3-cfb48b2d2de5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:52.879265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc537fdf-a615-4b5e-85c3-cfb48b2d2de5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:52.879627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-2ba42691-f6e4-4e90-97a2-6a9353eb5a4f None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:52.881694 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2ba42691-f6e4-4e90-97a2-6a9353eb5a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:52.890327 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 388/781] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 26 05:30:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:52.890327 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc537fdf-a615-4b5e-85c3-cfb48b2d2de5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:52.998671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 394/782] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:30:52.999890 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba42691-f6e4-4e90-97a2-6a9353eb5a4f None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:53.026607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc537fdf-a615-4b5e-85c3-cfb48b2d2de5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:53.046915 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:53.076281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3594e5-169d-47a9-bfce-573de6ca2e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:53.078083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3594e5-169d-47a9-bfce-573de6ca2e02 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:53.078444 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3594e5-169d-47a9-bfce-573de6ca2e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:53.078856 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3594e5-169d-47a9-bfce-573de6ca2e02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:53.087965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3594e5-169d-47a9-bfce-573de6ca2e02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:53.089614 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3594e5-169d-47a9-bfce-573de6ca2e02 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:53.106281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-dd3594e5-169d-47a9-bfce-573de6ca2e02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:53.106281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:53.106281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:53.106281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:53.106281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:53.106281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:53.106281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:53.106281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:53.106281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:53.106281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:53.106281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:53.106281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:53.106281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:53.106281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:53.106281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:53.107881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:53.107881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:53.107881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:53.107881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:53.107881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:53.107881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:53.107881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:53.107881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:53.107881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:53.107881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:53.107881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:53.107881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:53.107881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:53.107881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:53.107881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:53.107881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:53.107881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:53.110288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:53.110288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:53.110288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:53.110288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:53.110288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:53.110288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:53.110288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:53.110288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:53.112377 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 389/783] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:53.114046 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3594e5-169d-47a9-bfce-573de6ca2e02 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:53.131575 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:53.216697 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba42691-f6e4-4e90-97a2-6a9353eb5a4f None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:53.277832 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3594e5-169d-47a9-bfce-573de6ca2e02 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:53.280408 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99e348-0260-4604-b66a-f0ed0c9cf202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:53.281464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99e348-0260-4604-b66a-f0ed0c9cf202 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:53.281955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99e348-0260-4604-b66a-f0ed0c9cf202 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:53.282437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99e348-0260-4604-b66a-f0ed0c9cf202 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:53.283692 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99e348-0260-4604-b66a-f0ed0c9cf202 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:53.286311 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99e348-0260-4604-b66a-f0ed0c9cf202 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:53.322764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 395/784] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:53.323912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99e348-0260-4604-b66a-f0ed0c9cf202 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:53.354763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33e4c4-443e-4091-964c-ccecd36059b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:53.355619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33e4c4-443e-4091-964c-ccecd36059b5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:53.356100 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33e4c4-443e-4091-964c-ccecd36059b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:53.356612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33e4c4-443e-4091-964c-ccecd36059b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:53.357730 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33e4c4-443e-4091-964c-ccecd36059b5 None admin] RBAC: Adding query filter params (interface=public, service_id=ebc3df886d0e4c898428f94803f7a6b2, region_id=RegionOne) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:53.358229 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33e4c4-443e-4091-964c-ccecd36059b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:53.360973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33e4c4-443e-4091-964c-ccecd36059b5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:53.375899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 390/785] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Sep 26 05:30:53 2022] GET /identity/v3/endpoints?service_id=ebc3df886d0e4c898428f94803f7a6b2&interface=public®ion_id=RegionOne => generated 194 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:53.385752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33e4c4-443e-4091-964c-ccecd36059b5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:53.444491 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:53.456006 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99e348-0260-4604-b66a-f0ed0c9cf202 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:53.476489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:53.500023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d62b9d-7716-4992-88f0-1de6380d4384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:53.500649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d62b9d-7716-4992-88f0-1de6380d4384 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:53.501057 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d62b9d-7716-4992-88f0-1de6380d4384 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:53.501456 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d62b9d-7716-4992-88f0-1de6380d4384 None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:53.502173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d62b9d-7716-4992-88f0-1de6380d4384 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:53.502471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d62b9d-7716-4992-88f0-1de6380d4384 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:53.504016 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d62b9d-7716-4992-88f0-1de6380d4384 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:53.519136 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 396/786] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:53.520634 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb406b6d-2a90-445f-b3a2-be8c1e0a5093 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:53.520917 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb406b6d-2a90-445f-b3a2-be8c1e0a5093 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:53.521208 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb406b6d-2a90-445f-b3a2-be8c1e0a5093 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:53.522896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 397/787] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:53.523728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb406b6d-2a90-445f-b3a2-be8c1e0a5093 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:53.562752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33e4c4-443e-4091-964c-ccecd36059b5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:53.606324 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d1f3c-1bf2-4509-9bdd-5a33f49ae7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:53.606852 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d1f3c-1bf2-4509-9bdd-5a33f49ae7e5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:53.607132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d1f3c-1bf2-4509-9bdd-5a33f49ae7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:53.607395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d1f3c-1bf2-4509-9bdd-5a33f49ae7e5 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:53.608168 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d1f3c-1bf2-4509-9bdd-5a33f49ae7e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:53.609795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d1f3c-1bf2-4509-9bdd-5a33f49ae7e5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:53.616222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 391/788] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 26 05:30:53 2022] GET /identity/v3/services => generated 772 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:30:53.617618 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c2038-c722-4c5f-8370-92415141e4b1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:53.617902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c2038-c722-4c5f-8370-92415141e4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:53.618175 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c2038-c722-4c5f-8370-92415141e4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:53.639266 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f72c2038-c722-4c5f-8370-92415141e4b1 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:53.640691 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f72c2038-c722-4c5f-8370-92415141e4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:53.665366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb406b6d-2a90-445f-b3a2-be8c1e0a5093 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:53.725227 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113b414-aa36-46e3-8d31-dcc4ab7650f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:53.726320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113b414-aa36-46e3-8d31-dcc4ab7650f4 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:53.726320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113b414-aa36-46e3-8d31-dcc4ab7650f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:53.726496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113b414-aa36-46e3-8d31-dcc4ab7650f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:53.744362 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113b414-aa36-46e3-8d31-dcc4ab7650f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:53.747296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 392/789] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:30:53.748751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c2038-c722-4c5f-8370-92415141e4b1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:53.749752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-0113b414-aa36-46e3-8d31-dcc4ab7650f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:30:53.749752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:53.749752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:53.749752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:53.749752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:53.749752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:53.749752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:53.749752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:53.749752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:53.749752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:53.749752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:53.749752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:53.749752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:53.749752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:53.749752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:53.749752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:53.750710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:53.750710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:53.750710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:53.750710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:53.750710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:53.750710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:53.750710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:53.750710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:53.750710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:53.750710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:53.750710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:53.750710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:53.750710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:53.750710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:53.750710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:53.750710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:53.752002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:53.752002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:53.752002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:53.752002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:53.752002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:53.752002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:53.752002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:53.752002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:53.752002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:53.752002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:53.752002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:30:53.752002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:53.752002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 398/790] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:30:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:53.762379 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113b414-aa36-46e3-8d31-dcc4ab7650f4 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:53.820709 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:53.821470 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:53.945783 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113b414-aa36-46e3-8d31-dcc4ab7650f4 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:54.007819 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72c2038-c722-4c5f-8370-92415141e4b1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:54.009785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2f5e8-db00-4734-869f-43cf41f992d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:54.011194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2f5e8-db00-4734-869f-43cf41f992d5 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:54.011798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2f5e8-db00-4734-869f-43cf41f992d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:54.012276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2f5e8-db00-4734-869f-43cf41f992d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:54.013519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2f5e8-db00-4734-869f-43cf41f992d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:54.014057 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2f5e8-db00-4734-869f-43cf41f992d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:54.016842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2f5e8-db00-4734-869f-43cf41f992d5 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:54.035380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 399/791] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:30:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:54.044674 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2f5e8-db00-4734-869f-43cf41f992d5 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:54.060186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20cc7b3-1231-41dc-baeb-5e24de0a987e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:54.061420 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cc7b3-1231-41dc-baeb-5e24de0a987e None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:54.061917 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cc7b3-1231-41dc-baeb-5e24de0a987e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:54.062384 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cc7b3-1231-41dc-baeb-5e24de0a987e None admin] PATH_INFO: `/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/92556048cbd04c409c3f4b6fd645627f/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:54.066106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20cc7b3-1231-41dc-baeb-5e24de0a987e None admin] RBAC: Authorizing `identity:create_grant(project_id=5e44d601292b47b7b66d6f3efa80b450, user_id=92556048cbd04c409c3f4b6fd645627f, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:54.068443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20cc7b3-1231-41dc-baeb-5e24de0a987e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:54.099723 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 393/792] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:30:53 2022] PUT /identity/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/92556048cbd04c409c3f4b6fd645627f/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:54.131023 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20cc7b3-1231-41dc-baeb-5e24de0a987e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:54.296347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:54.318951 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20cc7b3-1231-41dc-baeb-5e24de0a987e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:54.357080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:54.360179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b259bd-d8d8-4e47-b9f4-76f31af4ab0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:54.361072 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b259bd-d8d8-4e47-b9f4-76f31af4ab0c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:54.361514 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b259bd-d8d8-4e47-b9f4-76f31af4ab0c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:54.361846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b259bd-d8d8-4e47-b9f4-76f31af4ab0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:54.366469 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2f5e8-db00-4734-869f-43cf41f992d5 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:54.375825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b259bd-d8d8-4e47-b9f4-76f31af4ab0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:54.380992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-80b259bd-d8d8-4e47-b9f4-76f31af4ab0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:54.380992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:54.380992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:54.380992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:54.380992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:54.380992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:54.380992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:54.380992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:54.380992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:54.380992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:54.380992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:54.380992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:54.380992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:54.380992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:54.380992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:54.380992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:54.382035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:54.382035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:54.382035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:54.382035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:54.382035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:54.382035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:54.382035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:54.382035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:54.382035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:54.382035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:54.382035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:54.382035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:54.382035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:54.382035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:54.382035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:54.382035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:54.383408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:54.383408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:54.383408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:54.383408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:54.383408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:54.383408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:54.383408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:54.383408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:54.383408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:54.383408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:54.383408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:30:54.383408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:54.383408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 394/793] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:30:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:30:54.394082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b259bd-d8d8-4e47-b9f4-76f31af4ab0c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:54.438014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0fb29-7731-49de-a2fc-6cd5eee588a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:54.439182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0fb29-7731-49de-a2fc-6cd5eee588a6 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:54.439738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0fb29-7731-49de-a2fc-6cd5eee588a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:54.440277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0fb29-7731-49de-a2fc-6cd5eee588a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:54.455987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0fb29-7731-49de-a2fc-6cd5eee588a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:54.457790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0fb29-7731-49de-a2fc-6cd5eee588a6 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:54.475711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-b1a0fb29-7731-49de-a2fc-6cd5eee588a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:54.475711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:54.475711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:54.475711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:54.475711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:54.475711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:54.475711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:54.475711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:54.475711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:54.475711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:54.475711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:54.475711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:54.475711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:54.475711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:54.475711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:54.475711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:54.476961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:54.476961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:54.476961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:54.476961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:54.476961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:54.476961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:54.476961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:54.476961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:54.476961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:54.476961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:54.476961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:54.476961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:54.476961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:54.476961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:54.476961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:54.476961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:54.476961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:54.478147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:54.478147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:54.478147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:54.480936 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 400/794] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 437 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:54.488941 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0fb29-7731-49de-a2fc-6cd5eee588a6 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:54.570400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b259bd-d8d8-4e47-b9f4-76f31af4ab0c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:54.621332 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b3db3-0742-448b-b279-d547b2f26946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:54.621964 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b3db3-0742-448b-b279-d547b2f26946 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:54.622240 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b3db3-0742-448b-b279-d547b2f26946 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:54.622510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b3db3-0742-448b-b279-d547b2f26946 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:54.623184 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b3db3-0742-448b-b279-d547b2f26946 None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:54.623487 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b3db3-0742-448b-b279-d547b2f26946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:54.624914 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b3db3-0742-448b-b279-d547b2f26946 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:54.646055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 395/795] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:30:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:30:54.652176 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b3db3-0742-448b-b279-d547b2f26946 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:54.672513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:54.777712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0fb29-7731-49de-a2fc-6cd5eee588a6 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:54.898230 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc5596-1c3d-488f-9393-3fa615cdab2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:54.899192 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc5596-1c3d-488f-9393-3fa615cdab2b None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:54.899626 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc5596-1c3d-488f-9393-3fa615cdab2b None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:54.900053 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc5596-1c3d-488f-9393-3fa615cdab2b None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:54.900800 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc5596-1c3d-488f-9393-3fa615cdab2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:54.901279 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc5596-1c3d-488f-9393-3fa615cdab2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:54.902957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc5596-1c3d-488f-9393-3fa615cdab2b None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:54.930972 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b3db3-0742-448b-b279-d547b2f26946 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:54.937182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 401/796] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:54.955402 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc5596-1c3d-488f-9393-3fa615cdab2b None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:55.001852 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:55.032438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61b61a-b464-4b7c-bd04-eb6f68d70662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:55.034712 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61b61a-b464-4b7c-bd04-eb6f68d70662 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:55.038109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61b61a-b464-4b7c-bd04-eb6f68d70662 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:55.038967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61b61a-b464-4b7c-bd04-eb6f68d70662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:55.080331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61b61a-b464-4b7c-bd04-eb6f68d70662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:55.083809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61b61a-b464-4b7c-bd04-eb6f68d70662 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:55.129805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-ee61b61a-b464-4b7c-bd04-eb6f68d70662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:55.129805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:55.129805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:55.129805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:55.129805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:55.129805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:55.129805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:55.129805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:55.129805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:55.129805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:55.129805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:55.129805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:55.129805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:55.129805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:55.129805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:55.129805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:55.131612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:55.131612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:55.131612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:55.131612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:55.131612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:55.131612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:55.131612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:55.131612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:55.131612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:55.131612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:55.131612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:55.131612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:55.131612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:55.131612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:55.131612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:55.131612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:55.131612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:55.133344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:55.133344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:55.133344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:55.141395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 396/797] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 490 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:55.143337 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61b61a-b464-4b7c-bd04-eb6f68d70662 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:55.287530 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc5596-1c3d-488f-9393-3fa615cdab2b None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:55.360994 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985d1f1-02c0-4cdd-87b1-fe26ff0ea2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:55.362109 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985d1f1-02c0-4cdd-87b1-fe26ff0ea2ca None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:55.362603 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985d1f1-02c0-4cdd-87b1-fe26ff0ea2ca None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:55.363062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985d1f1-02c0-4cdd-87b1-fe26ff0ea2ca None admin] PATH_INFO: `/v3/users/swift` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:55.369749 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:55.382549 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b985d1f1-02c0-4cdd-87b1-fe26ff0ea2ca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:55.384044 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b985d1f1-02c0-4cdd-87b1-fe26ff0ea2ca None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:55.415989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-b985d1f1-02c0-4cdd-87b1-fe26ff0ea2ca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:30:55.415989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:55.415989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:55.415989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:55.415989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:55.415989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:55.415989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:55.415989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:55.415989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:55.415989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:55.415989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:55.415989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:55.415989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:55.415989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:55.415989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:55.415989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:55.417770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:55.419489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:55.419489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:55.419489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:55.419489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:55.419489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:55.419489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:55.419489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:55.419489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:55.419489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:30:55.419489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:55.428827 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 402/798] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:30:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 475 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:55.440760 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985d1f1-02c0-4cdd-87b1-fe26ff0ea2ca None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:55.513525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61b61a-b464-4b7c-bd04-eb6f68d70662 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:55.578049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:55.606967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cc1b8-4e70-498d-a200-e571f69f4a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:55.607609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cc1b8-4e70-498d-a200-e571f69f4a8f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:55.607891 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cc1b8-4e70-498d-a200-e571f69f4a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:55.608163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cc1b8-4e70-498d-a200-e571f69f4a8f None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:55.608820 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cc1b8-4e70-498d-a200-e571f69f4a8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:55.609122 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cc1b8-4e70-498d-a200-e571f69f4a8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:55.610548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cc1b8-4e70-498d-a200-e571f69f4a8f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:55.615609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985d1f1-02c0-4cdd-87b1-fe26ff0ea2ca None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:55.643507 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 397/799] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 501 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:55.652756 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baafafe-3932-4e1a-a08d-c253222fcc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:55.653422 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baafafe-3932-4e1a-a08d-c253222fcc88 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:55.653751 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baafafe-3932-4e1a-a08d-c253222fcc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:55.654023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baafafe-3932-4e1a-a08d-c253222fcc88 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:55.654263 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cc1b8-4e70-498d-a200-e571f69f4a8f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:55.654877 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baafafe-3932-4e1a-a08d-c253222fcc88 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:55.655228 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baafafe-3932-4e1a-a08d-c253222fcc88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:55.656627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baafafe-3932-4e1a-a08d-c253222fcc88 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:55.676684 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 403/800] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Sep 26 05:30:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:55.685737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baafafe-3932-4e1a-a08d-c253222fcc88 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:55.794735 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baafafe-3932-4e1a-a08d-c253222fcc88 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:55.824860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:55.835809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb37fc-8424-4685-940f-e617be6a2a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:55.836833 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb37fc-8424-4685-940f-e617be6a2a05 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:55.837390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb37fc-8424-4685-940f-e617be6a2a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:55.837872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb37fc-8424-4685-940f-e617be6a2a05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:55.850296 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb37fc-8424-4685-940f-e617be6a2a05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:55.852135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb37fc-8424-4685-940f-e617be6a2a05 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:55.880540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-b7eb37fc-8424-4685-940f-e617be6a2a05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:55.880540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:55.880540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:55.880540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:55.880540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:55.880540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:55.880540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:55.880540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:55.880540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:55.880540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:55.880540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:55.880540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:55.880540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:55.880540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:55.880540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:55.880540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:55.882383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:55.882383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:55.882383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:55.882383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:55.882383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:55.882383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:55.882383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:55.882383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:55.882383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:55.882383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:55.882383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:55.882383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:55.882383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:55.882383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:55.882383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:55.882383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:55.882383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:55.883971 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:55.883971 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:55.883971 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:55.887734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 404/801] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:30:55.896328 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cc1b8-4e70-498d-a200-e571f69f4a8f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:55.922729 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb37fc-8424-4685-940f-e617be6a2a05 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:55.961604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:55.976395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16dcb3-4a43-4cf8-8453-f2c4a2028faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:55.977674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16dcb3-4a43-4cf8-8453-f2c4a2028faa None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:55.978408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16dcb3-4a43-4cf8-8453-f2c4a2028faa None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:55.978898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16dcb3-4a43-4cf8-8453-f2c4a2028faa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:55.998443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16dcb3-4a43-4cf8-8453-f2c4a2028faa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:56.000278 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16dcb3-4a43-4cf8-8453-f2c4a2028faa None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:56.027051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-1e16dcb3-4a43-4cf8-8453-f2c4a2028faa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:30:56.027051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:56.027051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:56.027051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:56.027051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:56.027051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:56.027051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:56.027051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:56.027051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:56.027051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:56.027051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:56.027051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:56.027051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:56.027051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:56.027051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:56.027051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:56.028101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:56.029055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:56.029055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:56.029055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:56.029055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:56.029055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:56.029055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:56.029055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:56.029055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:56.029055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:30:56.029055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:56.033183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 398/802] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 26 05:30:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:56.034594 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16dcb3-4a43-4cf8-8453-f2c4a2028faa None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:56.184375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb37fc-8424-4685-940f-e617be6a2a05 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:56.252071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16dcb3-4a43-4cf8-8453-f2c4a2028faa None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:56.262843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f28e52-5b09-47da-96ee-28489fbf2780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:56.263487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f28e52-5b09-47da-96ee-28489fbf2780 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:56.263769 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f28e52-5b09-47da-96ee-28489fbf2780 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:56.264049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f28e52-5b09-47da-96ee-28489fbf2780 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:56.264716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f28e52-5b09-47da-96ee-28489fbf2780 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:56.265021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f28e52-5b09-47da-96ee-28489fbf2780 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:56.266451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f28e52-5b09-47da-96ee-28489fbf2780 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:56.302652 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce7e9a-d374-4493-a523-b6c74632f3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:56.302874 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 405/803] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:56.303641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce7e9a-d374-4493-a523-b6c74632f3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:56.303922 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce7e9a-d374-4493-a523-b6c74632f3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:56.304208 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce7e9a-d374-4493-a523-b6c74632f3d4 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:56.305084 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce7e9a-d374-4493-a523-b6c74632f3d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:56.305429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce7e9a-d374-4493-a523-b6c74632f3d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:56.307013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce7e9a-d374-4493-a523-b6c74632f3d4 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:56.311842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f28e52-5b09-47da-96ee-28489fbf2780 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:56.333298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 399/804] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Sep 26 05:30:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:56.335369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce7e9a-d374-4493-a523-b6c74632f3d4 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:56.434646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:56.439323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:56.492587 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f28e52-5b09-47da-96ee-28489fbf2780 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:56.536079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6541a27b-0b1c-40b2-801f-2af1237e42be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:56.536814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541a27b-0b1c-40b2-801f-2af1237e42be None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:56.537090 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541a27b-0b1c-40b2-801f-2af1237e42be None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:56.537400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541a27b-0b1c-40b2-801f-2af1237e42be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:56.545910 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6541a27b-0b1c-40b2-801f-2af1237e42be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:56.547407 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6541a27b-0b1c-40b2-801f-2af1237e42be None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:56.561110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-6541a27b-0b1c-40b2-801f-2af1237e42be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:56.561110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:56.561110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:56.561110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:56.561110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:56.561110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:56.561110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:56.561110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:56.561110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:56.561110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:56.561110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:56.561110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:56.561110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:56.561110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:56.561110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:56.562430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:56.562430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:56.562430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:56.562430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:56.562430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:56.562430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:56.562430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:56.562430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:56.562430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:56.562430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:56.562430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:56.562430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:56.562430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:56.562430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:56.562430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:56.562430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:56.562430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:56.563367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:56.563367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:56.563367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:56.563367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:56.563367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:56.563367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:56.563367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:56.563367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:56.563367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 406/805] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:30:56.570417 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6541a27b-0b1c-40b2-801f-2af1237e42be None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:56.704648 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce7e9a-d374-4493-a523-b6c74632f3d4 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:56.814021 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af94a9-34c9-4299-80d5-f042c2bb5c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:56.814701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af94a9-34c9-4299-80d5-f042c2bb5c73 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:56.814980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af94a9-34c9-4299-80d5-f042c2bb5c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:56.815254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af94a9-34c9-4299-80d5-f042c2bb5c73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:56.828219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af94a9-34c9-4299-80d5-f042c2bb5c73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:56.830128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af94a9-34c9-4299-80d5-f042c2bb5c73 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:56.844626 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:56.856698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6541a27b-0b1c-40b2-801f-2af1237e42be None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:56.861831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b7af94a9-34c9-4299-80d5-f042c2bb5c73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:56.861831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:56.861831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:56.861831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:56.861831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:56.861831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:56.861831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:56.861831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:56.861831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:56.861831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:56.861831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:56.861831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:56.861831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:56.861831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:56.861831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:56.861831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:56.864038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:56.864038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:56.864038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:56.864038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:56.864038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:56.864038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:56.864038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:56.864038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:56.864038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:56.864038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:56.864038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:56.864038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:56.864038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:56.864038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:56.864038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:56.864038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:56.864038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:56.866044 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:56.866044 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:56.866044 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:56.867654 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 400/806] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 529 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:30:56.869208 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b963b-8db2-4e84-aa1d-f5a9ff2e1fb7 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:56.869500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b963b-8db2-4e84-aa1d-f5a9ff2e1fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:56.869781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b963b-8db2-4e84-aa1d-f5a9ff2e1fb7 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:56.876018 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 401/807] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:56.877780 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b963b-8db2-4e84-aa1d-f5a9ff2e1fb7 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:56.927410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53448f41-10f6-4200-9f68-d758d94fc3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:56.928211 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53448f41-10f6-4200-9f68-d758d94fc3ba None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:56.928590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53448f41-10f6-4200-9f68-d758d94fc3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:56.928964 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53448f41-10f6-4200-9f68-d758d94fc3ba None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:56.930043 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53448f41-10f6-4200-9f68-d758d94fc3ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:56.930365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53448f41-10f6-4200-9f68-d758d94fc3ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:56.932276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53448f41-10f6-4200-9f68-d758d94fc3ba None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:56.947226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 407/808] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:56.948848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cf313-9a52-4030-aa46-77d497e0e114 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:56.949269 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cf313-9a52-4030-aa46-77d497e0e114 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:56.949648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cf313-9a52-4030-aa46-77d497e0e114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:56.970093 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-219cf313-9a52-4030-aa46-77d497e0e114 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:56.971911 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-219cf313-9a52-4030-aa46-77d497e0e114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:57.031362 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:57.046630 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b963b-8db2-4e84-aa1d-f5a9ff2e1fb7 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:57.064477 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 408/809] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:30:57.065823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219cf313-9a52-4030-aa46-77d497e0e114 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:57.100714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7956f2c-d094-470e-b470-f4b67bc89de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:57.101397 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7956f2c-d094-470e-b470-f4b67bc89de7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:57.101711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7956f2c-d094-470e-b470-f4b67bc89de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:57.102483 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7956f2c-d094-470e-b470-f4b67bc89de7 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:57.103268 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7956f2c-d094-470e-b470-f4b67bc89de7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:57.103591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7956f2c-d094-470e-b470-f4b67bc89de7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:57.105889 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7956f2c-d094-470e-b470-f4b67bc89de7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:57.129373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 402/810] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:57.129373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3377-9275-4005-92af-7ecf61b32c14 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:57.129373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3377-9275-4005-92af-7ecf61b32c14 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:57.129373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3377-9275-4005-92af-7ecf61b32c14 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:57.129373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 403/811] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:57.133901 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c3377-9275-4005-92af-7ecf61b32c14 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:57.197601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:57.210291 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219cf313-9a52-4030-aa46-77d497e0e114 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:57.261593 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167092b7-2cd4-457c-b9ba-8844c50ded9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:57.262294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167092b7-2cd4-457c-b9ba-8844c50ded9f None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:57.262572 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167092b7-2cd4-457c-b9ba-8844c50ded9f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:57.262846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167092b7-2cd4-457c-b9ba-8844c50ded9f None admin] PATH_INFO: `/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/00f258aba84d4dd59ebe8de973ac65c6/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:57.266124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167092b7-2cd4-457c-b9ba-8844c50ded9f None admin] RBAC: Authorizing `identity:create_grant(project_id=5e44d601292b47b7b66d6f3efa80b450, user_id=00f258aba84d4dd59ebe8de973ac65c6, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:57.267849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167092b7-2cd4-457c-b9ba-8844c50ded9f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:57.276076 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c3377-9275-4005-92af-7ecf61b32c14 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:57.290656 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 409/812] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:30:57 2022] PUT /identity/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/00f258aba84d4dd59ebe8de973ac65c6/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:57.292815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ea2e2-8241-4dcb-b54f-79941b918533 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:57.293476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ea2e2-8241-4dcb-b54f-79941b918533 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:57.293966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ea2e2-8241-4dcb-b54f-79941b918533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:57.315099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-219ea2e2-8241-4dcb-b54f-79941b918533 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:57.316966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-219ea2e2-8241-4dcb-b54f-79941b918533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:57.330272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb236e83-a856-4be9-9655-57f27bfc55ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:57.331037 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb236e83-a856-4be9-9655-57f27bfc55ef None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:57.331318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb236e83-a856-4be9-9655-57f27bfc55ef None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:57.331591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb236e83-a856-4be9-9655-57f27bfc55ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:57.343960 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb236e83-a856-4be9-9655-57f27bfc55ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:57.347880 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb236e83-a856-4be9-9655-57f27bfc55ef None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:57.379303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-cb236e83-a856-4be9-9655-57f27bfc55ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:57.379303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:57.379303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:57.379303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:57.379303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:57.379303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:57.379303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:57.379303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:57.379303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:57.379303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:57.379303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:57.379303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:57.379303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:30:57.379303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:30:57.379303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:30:57.380325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:30:57.380325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:57.380325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:57.380325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:57.380325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:57.380325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:57.380325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:57.380325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:57.380325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:57.380325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:57.380325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:57.380325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:57.380325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:57.380325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:57.380325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:57.380325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:57.380325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:57.381264 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:30:57.381264 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:30:57.381264 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:30:57.381264 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:30:57.381264 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:30:57.381264 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:30:57.381264 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:30:57.381264 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:57.382311 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 404/813] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:57.386512 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c714e7-86a0-4df8-87ed-578aa2828d4b None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:57.387010 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c714e7-86a0-4df8-87ed-578aa2828d4b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:57.389255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c714e7-86a0-4df8-87ed-578aa2828d4b None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:57.391578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 405/814] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:30:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:30:57.397284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c714e7-86a0-4df8-87ed-578aa2828d4b None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:57.421959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:57.458906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 410/815] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:30:57.462373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f985a3-eb76-4d45-8f01-684316edbddf None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:57.463006 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f985a3-eb76-4d45-8f01-684316edbddf None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:57.463620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f985a3-eb76-4d45-8f01-684316edbddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:57.511398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-46f985a3-eb76-4d45-8f01-684316edbddf None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:30:57.513458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-46f985a3-eb76-4d45-8f01-684316edbddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:30:57.609758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c714e7-86a0-4df8-87ed-578aa2828d4b None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:57.636214 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 411/816] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:30:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:30:57.642980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f985a3-eb76-4d45-8f01-684316edbddf None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:57.665541 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:57.686618 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046334c0-e393-4619-aba8-a82e8f8b2c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:57.690748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046334c0-e393-4619-aba8-a82e8f8b2c81 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:57.691254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046334c0-e393-4619-aba8-a82e8f8b2c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:57.693785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046334c0-e393-4619-aba8-a82e8f8b2c81 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:57.695308 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046334c0-e393-4619-aba8-a82e8f8b2c81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:57.698705 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046334c0-e393-4619-aba8-a82e8f8b2c81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:57.705337 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046334c0-e393-4619-aba8-a82e8f8b2c81 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:57.741893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 406/817] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:30:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:57.749428 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046334c0-e393-4619-aba8-a82e8f8b2c81 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:57.802149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f985a3-eb76-4d45-8f01-684316edbddf None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:57.838674 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb84429-160e-4c3b-8fca-2f38352a3b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:57.839460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb84429-160e-4c3b-8fca-2f38352a3b62 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:57.839735 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb84429-160e-4c3b-8fca-2f38352a3b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:57.840005 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb84429-160e-4c3b-8fca-2f38352a3b62 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:57.840598 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb84429-160e-4c3b-8fca-2f38352a3b62 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:57.842014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb84429-160e-4c3b-8fca-2f38352a3b62 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:57.851882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-abb84429-160e-4c3b-8fca-2f38352a3b62 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:30:57.851882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:57.851882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:30:57.851882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:30:57.851882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:30:57.851882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:30:57.851882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:30:57.851882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:30:57.851882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:30:57.851882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:57.851882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:30:57.851882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:57.851882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:57.851882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:57.851882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:57.851882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:57.851882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:57.853888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:57.853888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:57.853888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:57.853888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:57.853888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:57.853888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:57.853888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:30:57.853888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:30:57.853888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:30:57.853888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:30:57.853888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:57.853888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:57.853888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:57.853888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:57.853888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:57.853888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:57.855995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:57.855995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:57.855995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:57.855995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:57.855995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:57.855995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:57.855995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:57.855995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:57.855995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:57.855995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:57.855995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:30:57.855995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:30:57.855995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:30:57.855995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:57.855995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 412/818] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:30:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:57.865494 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb84429-160e-4c3b-8fca-2f38352a3b62 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:57.917690 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:57.944254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046334c0-e393-4619-aba8-a82e8f8b2c81 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:57.967812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:57.990624 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b7bbef-2263-4c54-b432-df9067df034f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:57.991276 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7bbef-2263-4c54-b432-df9067df034f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:57.991568 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7bbef-2263-4c54-b432-df9067df034f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:57.991823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7bbef-2263-4c54-b432-df9067df034f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:58.003736 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b7bbef-2263-4c54-b432-df9067df034f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:30:58.009080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-a7b7bbef-2263-4c54-b432-df9067df034f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:30:58.009080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:58.009080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:58.009080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:58.009080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:58.009080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:58.009080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:58.009080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:58.009080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:58.009080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:58.009080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:58.009080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:58.009080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:30:58.009080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:30:58.009080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:30:58.009080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:30:58.010083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:30:58.010083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:30:58.010083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:30:58.010083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:30:58.010083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:58.010083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:58.010083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:58.010083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:58.010083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:58.010083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:58.010083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:58.010083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:58.010083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:58.010083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:58.010083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:58.010083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:58.011005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:58.011005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:58.011005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:58.011005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:58.011005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:30:58.011005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:30:58.011005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:30:58.011005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:30:58.011005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:30:58.011005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:30:58.011005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:30:58.011005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:30:58.011005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 407/819] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:30:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:58.011714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b7bbef-2263-4c54-b432-df9067df034f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:58.019554 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb84429-160e-4c3b-8fca-2f38352a3b62 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:58.068560 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054b36c-5225-4f5d-989c-1e56c7ef19a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:58.069656 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054b36c-5225-4f5d-989c-1e56c7ef19a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:58.070312 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054b36c-5225-4f5d-989c-1e56c7ef19a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:58.070595 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054b36c-5225-4f5d-989c-1e56c7ef19a9 None admin] PATH_INFO: `/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/0216bf67389e4627bf953b7dcc97a88c/roles/881210e567974dc0aa3043393c00a838` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:58.093999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054b36c-5225-4f5d-989c-1e56c7ef19a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e44d601292b47b7b66d6f3efa80b450, user_id=0216bf67389e4627bf953b7dcc97a88c, role_id=881210e567974dc0aa3043393c00a838)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:58.096919 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054b36c-5225-4f5d-989c-1e56c7ef19a9 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:58.120614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 413/820] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:30:57 2022] PUT /identity/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/0216bf67389e4627bf953b7dcc97a88c/roles/881210e567974dc0aa3043393c00a838 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:30:58.121946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054b36c-5225-4f5d-989c-1e56c7ef19a9 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:58.126422 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b7bbef-2263-4c54-b432-df9067df034f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:58.163398 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:58.172138 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72141c2-899e-4c7c-8ec8-35a1bc8fd3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:58.172783 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72141c2-899e-4c7c-8ec8-35a1bc8fd3ec None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:58.173074 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72141c2-899e-4c7c-8ec8-35a1bc8fd3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:58.173385 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72141c2-899e-4c7c-8ec8-35a1bc8fd3ec None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:58.174098 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72141c2-899e-4c7c-8ec8-35a1bc8fd3ec None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:58.174483 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72141c2-899e-4c7c-8ec8-35a1bc8fd3ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:58.176027 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72141c2-899e-4c7c-8ec8-35a1bc8fd3ec None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:58.186310 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 408/821] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:30:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:58.186310 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72141c2-899e-4c7c-8ec8-35a1bc8fd3ec None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:58.294715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054b36c-5225-4f5d-989c-1e56c7ef19a9 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:58.344856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:58.367983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5947792-611d-4358-af33-67d9e37f42f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:58.368971 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5947792-611d-4358-af33-67d9e37f42f3 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:58.369431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5947792-611d-4358-af33-67d9e37f42f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:58.369843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5947792-611d-4358-af33-67d9e37f42f3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:58.370973 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5947792-611d-4358-af33-67d9e37f42f3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:58.373522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5947792-611d-4358-af33-67d9e37f42f3 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:58.377105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72141c2-899e-4c7c-8ec8-35a1bc8fd3ec None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:58.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-e5947792-611d-4358-af33-67d9e37f42f3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:30:58.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:58.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:30:58.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:30:58.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:30:58.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:30:58.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:30:58.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:30:58.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:30:58.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:58.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:30:58.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:58.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:58.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:58.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:58.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:58.387868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:58.389590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:58.389590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:58.389590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:58.389590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:58.389590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:58.389590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:58.389590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:30:58.389590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:30:58.389590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:30:58.389590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:30:58.389590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:58.389590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:58.389590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:58.389590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:58.389590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:58.389590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:58.391187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:58.391187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:58.391187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:58.391187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:58.391187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:58.391187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:58.391187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:58.391187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:58.391187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:58.391187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:58.391187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:30:58.391187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:30:58.391187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 05:30:58.391187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:58.400600 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 414/822] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:30:58 2022] GET /identity/v3/services/network => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:58.400600 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5947792-611d-4358-af33-67d9e37f42f3 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:58.441523 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ee92f-7f23-4014-9960-0e062b3b4f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:58.442039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ee92f-7f23-4014-9960-0e062b3b4f7c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:58.442314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ee92f-7f23-4014-9960-0e062b3b4f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:58.442581 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ee92f-7f23-4014-9960-0e062b3b4f7c None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:58.443257 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ee92f-7f23-4014-9960-0e062b3b4f7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:58.443581 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ee92f-7f23-4014-9960-0e062b3b4f7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:58.444975 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ee92f-7f23-4014-9960-0e062b3b4f7c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:58.455272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 409/823] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:30:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:58.456169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ee92f-7f23-4014-9960-0e062b3b4f7c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:58.541667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:58.602023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5947792-611d-4358-af33-67d9e37f42f3 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:58.626913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ee92f-7f23-4014-9960-0e062b3b4f7c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:58.670680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8a39f-b67d-48ec-a715-9ae445b97ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:58.671998 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8a39f-b67d-48ec-a715-9ae445b97ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:58.672682 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8a39f-b67d-48ec-a715-9ae445b97ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:58.673465 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8a39f-b67d-48ec-a715-9ae445b97ce9 None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:58.674764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8a39f-b67d-48ec-a715-9ae445b97ce9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:58.675461 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8a39f-b67d-48ec-a715-9ae445b97ce9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:58.678233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8a39f-b67d-48ec-a715-9ae445b97ce9 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:58.687507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 415/824] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:30:58 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:58.689502 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8a39f-b67d-48ec-a715-9ae445b97ce9 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:58.695266 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c012aea-96fe-408e-b050-eb8f564183fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:58.696585 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c012aea-96fe-408e-b050-eb8f564183fa None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:58.697227 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c012aea-96fe-408e-b050-eb8f564183fa None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:58.697861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c012aea-96fe-408e-b050-eb8f564183fa None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:58.699660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c012aea-96fe-408e-b050-eb8f564183fa None admin] RBAC: Adding query filter params (name=network) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:58.699660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c012aea-96fe-408e-b050-eb8f564183fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:58.702374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c012aea-96fe-408e-b050-eb8f564183fa None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:58.711795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 410/825] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 26 05:30:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:58.723040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c012aea-96fe-408e-b050-eb8f564183fa None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:58.795980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:58.905914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8a39f-b67d-48ec-a715-9ae445b97ce9 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:58.982180 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c012aea-96fe-408e-b050-eb8f564183fa None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:59.019389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94dbe3b-f15b-402b-8754-6b311ede9965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:59.020776 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:59.020958 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94dbe3b-f15b-402b-8754-6b311ede9965 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:59.021328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94dbe3b-f15b-402b-8754-6b311ede9965 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:59.021775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94dbe3b-f15b-402b-8754-6b311ede9965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:59.059978 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc9da6-9be2-451a-8bda-cf91b74c4324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:59.062516 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94dbe3b-f15b-402b-8754-6b311ede9965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:59.063241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc9da6-9be2-451a-8bda-cf91b74c4324 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:59.065746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc9da6-9be2-451a-8bda-cf91b74c4324 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:59.066506 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc9da6-9be2-451a-8bda-cf91b74c4324 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:59.067980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc9da6-9be2-451a-8bda-cf91b74c4324 None admin] RBAC: Adding query filter params (type=network) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:59.070172 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc9da6-9be2-451a-8bda-cf91b74c4324 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:59.070702 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94dbe3b-f15b-402b-8754-6b311ede9965 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:59.076542 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc9da6-9be2-451a-8bda-cf91b74c4324 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:59.093649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 411/826] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 26 05:30:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:30:59.102350 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc9da6-9be2-451a-8bda-cf91b74c4324 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:59.117539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-f94dbe3b-f15b-402b-8754-6b311ede9965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:59.117539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:59.117539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:59.117539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:59.117539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:59.117539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:59.117539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:59.117539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:59.117539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:59.117539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:59.117539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:59.117539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:59.117539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:30:59.117539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:30:59.117539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:30:59.117539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:30:59.119298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:59.119298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:59.119298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:59.119298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:59.119298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:59.119298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:59.119298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:59.119298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:59.119298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:59.119298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:59.119298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:59.119298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:59.119298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:59.119298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:59.119298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:59.119298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:59.119298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:30:59.121041 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:30:59.121041 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:30:59.121041 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:59.122812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 416/827] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:59.127594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94dbe3b-f15b-402b-8754-6b311ede9965 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:59.356087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94dbe3b-f15b-402b-8754-6b311ede9965 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:59.361641 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:30:59.373602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc9da6-9be2-451a-8bda-cf91b74c4324 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:59.427040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d3819-292e-4ed5-be83-fc37446efe6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:59.428016 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d3819-292e-4ed5-be83-fc37446efe6a None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:59.428485 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d3819-292e-4ed5-be83-fc37446efe6a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:59.428922 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d3819-292e-4ed5-be83-fc37446efe6a None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:59.430071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d3819-292e-4ed5-be83-fc37446efe6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:30:59.430603 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d3819-292e-4ed5-be83-fc37446efe6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:59.433249 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d3819-292e-4ed5-be83-fc37446efe6a None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:59.458782 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 417/828] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:30:59.468453 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d3819-292e-4ed5-be83-fc37446efe6a None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:30:59.493313 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f9d11-3b6d-4c04-b424-03ad81c5dd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:59.494405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f9d11-3b6d-4c04-b424-03ad81c5dd57 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:59.494803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f9d11-3b6d-4c04-b424-03ad81c5dd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:59.495376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f9d11-3b6d-4c04-b424-03ad81c5dd57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:59.496569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f9d11-3b6d-4c04-b424-03ad81c5dd57 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:59.499185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f9d11-3b6d-4c04-b424-03ad81c5dd57 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:59.562774 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 412/829] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:30:59 2022] POST /identity/v3/endpoints => generated 337 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:30:59.759115 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d3819-292e-4ed5-be83-fc37446efe6a None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:30:59.868535 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4fbe3-987a-45ef-a459-96f91359081b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:30:59.869655 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4fbe3-987a-45ef-a459-96f91359081b None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:30:59.870201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4fbe3-987a-45ef-a459-96f91359081b None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:30:59.870695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4fbe3-987a-45ef-a459-96f91359081b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:30:59.885626 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4fbe3-987a-45ef-a459-96f91359081b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:30:59.888408 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4fbe3-987a-45ef-a459-96f91359081b None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:30:59.912912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-d8e4fbe3-987a-45ef-a459-96f91359081b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:30:59.912912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:30:59.912912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:30:59.912912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:30:59.912912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:30:59.912912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:30:59.912912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:30:59.912912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:30:59.912912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:30:59.912912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:30:59.912912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:30:59.912912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:30:59.912912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:30:59.912912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:30:59.912912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:30:59.912912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:30:59.914459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:30:59.915999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:30:59.915999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:30:59.915999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:30:59.915999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:30:59.915999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:30:59.915999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:30:59.915999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:30:59.915999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:30:59.915999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:30:59.915999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:30:59.915999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 418/830] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 26 05:30:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 448 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:30:59.924858 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f9d11-3b6d-4c04-b424-03ad81c5dd57 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:00.015174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:00.092348 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f9d11-3b6d-4c04-b424-03ad81c5dd57 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:00.138745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0254da0-8c73-4e5b-885a-d52266819da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:00.139619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0254da0-8c73-4e5b-885a-d52266819da5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:00.139899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0254da0-8c73-4e5b-885a-d52266819da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:00.140172 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0254da0-8c73-4e5b-885a-d52266819da5 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:00.140915 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0254da0-8c73-4e5b-885a-d52266819da5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:00.141272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0254da0-8c73-4e5b-885a-d52266819da5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:00.142739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0254da0-8c73-4e5b-885a-d52266819da5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:00.166626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 413/831] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 26 05:30:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:00.175397 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4fbe3-987a-45ef-a459-96f91359081b None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:00.247286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:00.400189 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4fbe3-987a-45ef-a459-96f91359081b None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:00.465550 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a71ed-45fa-4479-995a-b58d9e7b0eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:00.466957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a71ed-45fa-4479-995a-b58d9e7b0eaa None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:00.467666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a71ed-45fa-4479-995a-b58d9e7b0eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:00.468345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a71ed-45fa-4479-995a-b58d9e7b0eaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:00.601939 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a71ed-45fa-4479-995a-b58d9e7b0eaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:00.605051 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a71ed-45fa-4479-995a-b58d9e7b0eaa None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:00.647554 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:00.801658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 419/832] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:31:00 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:31:00.810067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0254da0-8c73-4e5b-885a-d52266819da5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:00.969921 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:00.974539 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0254da0-8c73-4e5b-885a-d52266819da5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:01.007612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c80b6-525a-49ee-b742-41888afe66e5 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:01.008417 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c80b6-525a-49ee-b742-41888afe66e5 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:01.008595 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c80b6-525a-49ee-b742-41888afe66e5 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:01.014141 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7742b15-2120-4b2c-adec-ce753103b145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:01.014801 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7742b15-2120-4b2c-adec-ce753103b145 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:01.015076 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7742b15-2120-4b2c-adec-ce753103b145 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:01.015344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7742b15-2120-4b2c-adec-ce753103b145 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:01.017684 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 420/833] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:01 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:01.020829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4818809-7862-4b2e-9d77-df4a5ed58179 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:01.021360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4818809-7862-4b2e-9d77-df4a5ed58179 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:01.021874 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4818809-7862-4b2e-9d77-df4a5ed58179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:01.030083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7742b15-2120-4b2c-adec-ce753103b145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:01.031624 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7742b15-2120-4b2c-adec-ce753103b145 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:01.054415 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-c4818809-7862-4b2e-9d77-df4a5ed58179 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:01.056331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c4818809-7862-4b2e-9d77-df4a5ed58179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:01.056611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b7742b15-2120-4b2c-adec-ce753103b145 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:01.056611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:01.056611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:01.056611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:01.056611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:01.056611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:01.056611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:01.056611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:01.056611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:01.056611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:01.056611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:01.056611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:01.056611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:01.056611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:01.056611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:01.056611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:01.065573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:01.065573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:01.065573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:01.065573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:01.065573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:01.065573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:01.065573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:01.065573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:01.065573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:01.065573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:01.065573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:01.065573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:01.065573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:01.065573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:01.065573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:01.065573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:01.065573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:01.067333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:01.067333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:01.067333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:01.067333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 414/834] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:01.067333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4523cd0-4957-4d33-8bf1-faf8b8562f95 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:01.067333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4523cd0-4957-4d33-8bf1-faf8b8562f95 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:01.067333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4523cd0-4957-4d33-8bf1-faf8b8562f95 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:01.067333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 415/835] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:01.071076 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4523cd0-4957-4d33-8bf1-faf8b8562f95 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:01.164998 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 421/836] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:31:01.167814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22998b3e-4e1f-443e-a09b-21b81822879b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:01.168541 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22998b3e-4e1f-443e-a09b-21b81822879b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:01.169228 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22998b3e-4e1f-443e-a09b-21b81822879b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:01.202339 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-22998b3e-4e1f-443e-a09b-21b81822879b None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:01.204717 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-22998b3e-4e1f-443e-a09b-21b81822879b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:01.261870 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4523cd0-4957-4d33-8bf1-faf8b8562f95 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:01.292757 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:01.345028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 422/837] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:01.346962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa22929-bc64-4fbb-bc09-b3202ac736a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:01.348085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa22929-bc64-4fbb-bc09-b3202ac736a0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:01.348560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa22929-bc64-4fbb-bc09-b3202ac736a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:01.349082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa22929-bc64-4fbb-bc09-b3202ac736a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:01.350373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa22929-bc64-4fbb-bc09-b3202ac736a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:01.350918 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa22929-bc64-4fbb-bc09-b3202ac736a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:01.353613 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa22929-bc64-4fbb-bc09-b3202ac736a0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:01.376630 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 416/838] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:01.385473 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22998b3e-4e1f-443e-a09b-21b81822879b None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:01.592689 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22998b3e-4e1f-443e-a09b-21b81822879b None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:01.640221 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa22929-bc64-4fbb-bc09-b3202ac736a0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:01.664514 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:01.666696 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643208da-5ad2-4116-9b49-fcb0b517da0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:01.668528 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643208da-5ad2-4116-9b49-fcb0b517da0c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:01.669245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643208da-5ad2-4116-9b49-fcb0b517da0c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:01.669965 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643208da-5ad2-4116-9b49-fcb0b517da0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:01.686386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643208da-5ad2-4116-9b49-fcb0b517da0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:01.690039 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643208da-5ad2-4116-9b49-fcb0b517da0c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:01.716429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-643208da-5ad2-4116-9b49-fcb0b517da0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:01.716429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:01.716429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:01.716429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:01.716429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:01.716429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:01.716429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:01.716429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:01.716429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:01.716429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:01.716429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:01.716429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:01.716429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:31:01.716429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:31:01.716429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:31:01.718472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:31:01.718472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:01.718472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:01.718472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:01.718472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:01.718472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:01.718472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:01.718472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:01.718472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:01.718472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:01.718472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:01.718472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:01.718472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:01.718472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:01.718472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:01.718472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:01.718472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:01.720546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:31:01.720546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:31:01.720546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:31:01.720546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:31:01.720546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:31:01.720546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:31:01.720546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:01.720546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:01.725604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 423/839] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:31:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:01.735354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643208da-5ad2-4116-9b49-fcb0b517da0c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:01.819456 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:01.850170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa22929-bc64-4fbb-bc09-b3202ac736a0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:01.911907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541a9f1-d2eb-420d-8d07-7911e634c4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:01.912733 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541a9f1-d2eb-420d-8d07-7911e634c4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:01.913010 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541a9f1-d2eb-420d-8d07-7911e634c4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:01.913362 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541a9f1-d2eb-420d-8d07-7911e634c4e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:01.931051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541a9f1-d2eb-420d-8d07-7911e634c4e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:31:01.937272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b541a9f1-d2eb-420d-8d07-7911e634c4e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:31:01.937272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:01.937272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:01.937272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:01.937272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:01.937272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:01.937272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:01.937272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:01.937272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:01.937272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:01.937272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:01.937272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:01.937272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:31:01.937272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:31:01.937272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:31:01.937272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:31:01.938301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:31:01.938301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:31:01.938301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:31:01.938301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:31:01.938301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:01.938301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:01.938301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:01.938301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:01.938301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:01.938301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:01.938301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:01.938301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:01.938301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:01.938301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:01.938301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:01.938301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:01.939153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:01.939153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:01.939153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:01.939153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:01.939153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:31:01.939153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:31:01.939153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:31:01.939153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:31:01.939153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:31:01.939153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:31:01.939153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:31:01.939153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:01.939153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 417/840] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:31:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:01.939870 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541a9f1-d2eb-420d-8d07-7911e634c4e0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:01.955130 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643208da-5ad2-4116-9b49-fcb0b517da0c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:02.017601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3aac37-51e6-418d-bce7-17161476fb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:02.018584 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aac37-51e6-418d-bce7-17161476fb55 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:02.018735 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aac37-51e6-418d-bce7-17161476fb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:02.019019 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3aac37-51e6-418d-bce7-17161476fb55 None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:02.020209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3aac37-51e6-418d-bce7-17161476fb55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:02.020694 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3aac37-51e6-418d-bce7-17161476fb55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:02.023081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3aac37-51e6-418d-bce7-17161476fb55 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:02.040423 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:02.044014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 424/841] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:31:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:02.044014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3aac37-51e6-418d-bce7-17161476fb55 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:02.101980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541a9f1-d2eb-420d-8d07-7911e634c4e0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:02.161033 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65237c60-1dfd-46e2-9575-a5f01ac40d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:02.161872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65237c60-1dfd-46e2-9575-a5f01ac40d45 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:02.162416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65237c60-1dfd-46e2-9575-a5f01ac40d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:02.162800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65237c60-1dfd-46e2-9575-a5f01ac40d45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:02.182253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65237c60-1dfd-46e2-9575-a5f01ac40d45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:31:02.190296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-65237c60-1dfd-46e2-9575-a5f01ac40d45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:31:02.190296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:02.190296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:02.190296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:02.190296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:02.190296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:02.190296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:02.190296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:02.190296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:02.190296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:02.190296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:02.190296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:02.190296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:31:02.190296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:31:02.190296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:31:02.190296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:31:02.192248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:31:02.192248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:31:02.192248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:31:02.192248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:31:02.192248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:02.192248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:02.192248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:02.192248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:02.192248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:02.192248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:02.192248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:02.192248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:02.192248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:02.192248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:02.192248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:02.192248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:02.194642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:02.194642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:02.194642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:02.194642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:02.194642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:31:02.194642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:31:02.194642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:31:02.194642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:31:02.194642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:31:02.194642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:31:02.194642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:31:02.194642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:02.194642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 418/842] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:31:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:02.194642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65237c60-1dfd-46e2-9575-a5f01ac40d45 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:02.237002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:02.241424 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3aac37-51e6-418d-bce7-17161476fb55 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:02.287698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd5541-266c-49da-b8bb-d998becb579f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:02.288322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd5541-266c-49da-b8bb-d998becb579f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:02.288601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd5541-266c-49da-b8bb-d998becb579f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:02.288878 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd5541-266c-49da-b8bb-d998becb579f None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:02.289604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd5541-266c-49da-b8bb-d998becb579f None admin] RBAC: Adding query filter params (name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:02.289910 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd5541-266c-49da-b8bb-d998becb579f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:02.291300 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd5541-266c-49da-b8bb-d998becb579f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:02.300251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 425/843] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:31:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:02.301121 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd5541-266c-49da-b8bb-d998becb579f None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:02.345754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65237c60-1dfd-46e2-9575-a5f01ac40d45 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:02.395134 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:02.402994 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7017d2-2f25-437c-be3e-fb1f5fedd97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:02.403760 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7017d2-2f25-437c-be3e-fb1f5fedd97b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:02.404129 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7017d2-2f25-437c-be3e-fb1f5fedd97b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:02.404488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7017d2-2f25-437c-be3e-fb1f5fedd97b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:02.405417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7017d2-2f25-437c-be3e-fb1f5fedd97b None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=92556048cbd04c409c3f4b6fd645627f) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:02.405830 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7017d2-2f25-437c-be3e-fb1f5fedd97b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:02.407402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7017d2-2f25-437c-be3e-fb1f5fedd97b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:02.413920 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd5541-266c-49da-b8bb-d998becb579f None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:02.414675 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 419/844] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:31:02 2022] GET /identity/v3/role_assignments?user.id=92556048cbd04c409c3f4b6fd645627f&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:02.416079 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf9610-af3e-4fc7-8b27-f4fcd15bb997 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:02.416364 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf9610-af3e-4fc7-8b27-f4fcd15bb997 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:02.416639 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf9610-af3e-4fc7-8b27-f4fcd15bb997 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:02.418395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 420/845] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:02.419182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf9610-af3e-4fc7-8b27-f4fcd15bb997 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:02.503869 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457fe0d-4d04-45e6-a4cd-7a8124e7df60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:02.504884 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457fe0d-4d04-45e6-a4cd-7a8124e7df60 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:02.505407 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457fe0d-4d04-45e6-a4cd-7a8124e7df60 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:02.505879 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457fe0d-4d04-45e6-a4cd-7a8124e7df60 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:02.507089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457fe0d-4d04-45e6-a4cd-7a8124e7df60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:02.507621 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457fe0d-4d04-45e6-a4cd-7a8124e7df60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:02.510266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457fe0d-4d04-45e6-a4cd-7a8124e7df60 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:02.529224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 426/846] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:31:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:02.529535 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d5fa0-423e-4b3a-bf64-edb335925f88 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:02.530176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d5fa0-423e-4b3a-bf64-edb335925f88 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:02.530332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d5fa0-423e-4b3a-bf64-edb335925f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:02.562443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-6b6d5fa0-423e-4b3a-bf64-edb335925f88 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:02.564596 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6b6d5fa0-423e-4b3a-bf64-edb335925f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:02.580738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf9610-af3e-4fc7-8b27-f4fcd15bb997 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:02.602856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:02.640596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573c3a0-ceb4-4302-81c0-6c42f6f780dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:02.641669 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573c3a0-ceb4-4302-81c0-6c42f6f780dd None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:02.642108 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573c3a0-ceb4-4302-81c0-6c42f6f780dd None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:02.642508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573c3a0-ceb4-4302-81c0-6c42f6f780dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:02.662122 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573c3a0-ceb4-4302-81c0-6c42f6f780dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:02.664759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573c3a0-ceb4-4302-81c0-6c42f6f780dd None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:02.688258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-7573c3a0-ceb4-4302-81c0-6c42f6f780dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:02.688258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:02.688258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:02.688258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:02.688258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:02.688258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:02.688258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:02.688258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:02.688258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:02.688258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:02.688258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:02.688258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:02.688258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:02.688258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:02.688258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:02.688258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:02.690314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:02.690314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:02.690314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:02.690314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:02.690314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:02.690314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:02.690314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:02.690314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:02.690314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:02.690314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:02.690314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:02.690314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:02.690314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:02.690314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:02.690314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:02.690314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:02.690314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:02.692522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:02.692522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:02.692522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:02.692522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 421/847] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:02.692522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573c3a0-ceb4-4302-81c0-6c42f6f780dd None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:02.710194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 427/848] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:02.711349 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d5fa0-423e-4b3a-bf64-edb335925f88 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:02.792178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:02.881279 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d5fa0-423e-4b3a-bf64-edb335925f88 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:02.907937 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573c3a0-ceb4-4302-81c0-6c42f6f780dd None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:02.949363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074cac1e-278b-4779-91f2-1a957d2bf405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:02.950874 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cac1e-278b-4779-91f2-1a957d2bf405 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:02.951666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cac1e-278b-4779-91f2-1a957d2bf405 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:02.952415 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cac1e-278b-4779-91f2-1a957d2bf405 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:02.953874 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074cac1e-278b-4779-91f2-1a957d2bf405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:02.954675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074cac1e-278b-4779-91f2-1a957d2bf405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:02.957532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074cac1e-278b-4779-91f2-1a957d2bf405 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:02.977016 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93808ef5-4ab8-416e-8a86-e8f9eccb96fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:02.978159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93808ef5-4ab8-416e-8a86-e8f9eccb96fb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:02.978675 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93808ef5-4ab8-416e-8a86-e8f9eccb96fb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:02.979166 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93808ef5-4ab8-416e-8a86-e8f9eccb96fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:02.982131 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 428/849] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:02.991798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074cac1e-278b-4779-91f2-1a957d2bf405 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:03.018903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:03.177094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93808ef5-4ab8-416e-8a86-e8f9eccb96fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:03.180139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93808ef5-4ab8-416e-8a86-e8f9eccb96fb None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:03.181436 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074cac1e-278b-4779-91f2-1a957d2bf405 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:03.209752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-93808ef5-4ab8-416e-8a86-e8f9eccb96fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:03.209752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:03.209752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:03.209752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:03.209752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:03.209752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:03.209752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:03.209752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:03.209752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:03.209752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:03.209752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:03.209752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:03.209752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:03.209752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:03.209752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:03.209752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:03.211561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:03.211561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:03.211561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:03.211561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:03.211561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:03.211561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:03.211561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:03.211561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:03.211561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:03.211561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:03.211561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:03.211561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:03.211561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:03.211561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:03.211561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:03.211561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:03.211561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:03.214246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:03.214246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:03.214246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:03.214246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 422/850] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 522 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:03.214246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93808ef5-4ab8-416e-8a86-e8f9eccb96fb None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:03.241483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6b33c-6cf7-4719-8581-234be8d76246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:03.242354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6b33c-6cf7-4719-8581-234be8d76246 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:03.242906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6b33c-6cf7-4719-8581-234be8d76246 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:03.243348 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6b33c-6cf7-4719-8581-234be8d76246 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:03.253978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6b33c-6cf7-4719-8581-234be8d76246 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:03.255577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6b33c-6cf7-4719-8581-234be8d76246 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:03.270138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-ccc6b33c-6cf7-4719-8581-234be8d76246 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:31:03.270138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:03.270138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:03.270138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:03.270138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:03.270138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:03.270138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:03.270138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:03.270138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:03.270138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:03.270138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:03.270138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:03.270138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:31:03.270138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:31:03.270138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:31:03.270138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:03.272002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:03.281048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:03.281048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:03.281048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:31:03.281048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:31:03.281048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:31:03.281048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:31:03.281048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:31:03.281048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:31:03.281048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:31:03.281048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:03.281048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 429/851] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 26 05:31:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:03.281048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6b33c-6cf7-4719-8581-234be8d76246 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:03.384212 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:03.390041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93808ef5-4ab8-416e-8a86-e8f9eccb96fb None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:03.414760 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:03.440718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7628b07-041a-4983-b0f2-43198a39d1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:03.442132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7628b07-041a-4983-b0f2-43198a39d1b5 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:03.442721 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7628b07-041a-4983-b0f2-43198a39d1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:03.443193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7628b07-041a-4983-b0f2-43198a39d1b5 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:03.444324 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7628b07-041a-4983-b0f2-43198a39d1b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:03.446916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7628b07-041a-4983-b0f2-43198a39d1b5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:03.451805 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6b33c-6cf7-4719-8581-234be8d76246 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:03.477414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 423/852] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 26 05:31:03 2022] POST /identity/v3/services => generated 251 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:31:03.477414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7628b07-041a-4983-b0f2-43198a39d1b5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:03.533241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea467e2-8b99-489e-a3ad-c1c6881cec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:03.534817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea467e2-8b99-489e-a3ad-c1c6881cec40 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:03.535559 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea467e2-8b99-489e-a3ad-c1c6881cec40 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:03.536452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea467e2-8b99-489e-a3ad-c1c6881cec40 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:03.538974 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea467e2-8b99-489e-a3ad-c1c6881cec40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:03.539774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea467e2-8b99-489e-a3ad-c1c6881cec40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:03.542790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea467e2-8b99-489e-a3ad-c1c6881cec40 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:03.568190 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 430/853] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:03.574300 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea467e2-8b99-489e-a3ad-c1c6881cec40 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:03.656493 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7628b07-041a-4983-b0f2-43198a39d1b5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:03.705182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90462586-8163-4277-9878-e5b18e1fc196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:03.707359 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90462586-8163-4277-9878-e5b18e1fc196 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:03.707947 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90462586-8163-4277-9878-e5b18e1fc196 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:03.708382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90462586-8163-4277-9878-e5b18e1fc196 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:03.709698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90462586-8163-4277-9878-e5b18e1fc196 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:03.710192 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90462586-8163-4277-9878-e5b18e1fc196 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:03.712629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90462586-8163-4277-9878-e5b18e1fc196 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:03.744194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea467e2-8b99-489e-a3ad-c1c6881cec40 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:03.749702 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 424/854] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Sep 26 05:31:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:03.753824 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90462586-8163-4277-9878-e5b18e1fc196 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:03.779625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:03.788474 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497f8ba-bfcf-48e8-a916-ed981455ab94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:03.789508 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497f8ba-bfcf-48e8-a916-ed981455ab94 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:03.790064 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497f8ba-bfcf-48e8-a916-ed981455ab94 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:03.790534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497f8ba-bfcf-48e8-a916-ed981455ab94 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:03.792196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:03.800006 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7497f8ba-bfcf-48e8-a916-ed981455ab94 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:03.801888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7497f8ba-bfcf-48e8-a916-ed981455ab94 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:03.829047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-7497f8ba-bfcf-48e8-a916-ed981455ab94 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:31:03.829047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:03.829047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:03.829047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:03.829047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:03.829047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:03.829047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:03.829047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:03.829047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:03.829047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:03.829047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:03.829047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:03.829047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:31:03.829047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:31:03.829047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:31:03.829047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:03.830184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:03.831216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:03.831216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:03.831216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:31:03.831216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:31:03.831216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:31:03.831216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:31:03.831216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:31:03.831216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:31:03.831216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 05:31:03.831216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:03.837670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 431/855] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:31:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:03.841300 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497f8ba-bfcf-48e8-a916-ed981455ab94 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:03.923037 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90462586-8163-4277-9878-e5b18e1fc196 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:03.968067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d358b61-2c19-407f-b485-290e14783f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:03.969407 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d358b61-2c19-407f-b485-290e14783f42 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:03.969896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d358b61-2c19-407f-b485-290e14783f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:03.970859 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d358b61-2c19-407f-b485-290e14783f42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:04.045709 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497f8ba-bfcf-48e8-a916-ed981455ab94 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:04.062739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d358b61-2c19-407f-b485-290e14783f42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:04.064626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d358b61-2c19-407f-b485-290e14783f42 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:04.093041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:04.107028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede78fdb-0f6c-4090-bd4e-b5945d37fd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:04.108733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede78fdb-0f6c-4090-bd4e-b5945d37fd18 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:04.109340 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede78fdb-0f6c-4090-bd4e-b5945d37fd18 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:04.109854 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede78fdb-0f6c-4090-bd4e-b5945d37fd18 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:04.111189 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede78fdb-0f6c-4090-bd4e-b5945d37fd18 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:04.111767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede78fdb-0f6c-4090-bd4e-b5945d37fd18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:04.113766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede78fdb-0f6c-4090-bd4e-b5945d37fd18 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:04.137883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 432/856] 10.222.0.57 () {56 vars in 1135 bytes} [Mon Sep 26 05:31:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:04.150293 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede78fdb-0f6c-4090-bd4e-b5945d37fd18 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:04.219298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:04.260247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 425/857] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:31:03 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 05:31:04.272116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d358b61-2c19-407f-b485-290e14783f42 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:04.284619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede78fdb-0f6c-4090-bd4e-b5945d37fd18 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:04.339137 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72ae49-a659-4915-a0d0-ed2226d06516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:04.340020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72ae49-a659-4915-a0d0-ed2226d06516 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:04.340314 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72ae49-a659-4915-a0d0-ed2226d06516 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:04.340802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72ae49-a659-4915-a0d0-ed2226d06516 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:04.415374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d358b61-2c19-407f-b485-290e14783f42 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:04.427868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72ae49-a659-4915-a0d0-ed2226d06516 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:04.429956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72ae49-a659-4915-a0d0-ed2226d06516 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:04.438621 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:04.464106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5de06-e649-4b26-a009-15459c75cac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:04.465965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5de06-e649-4b26-a009-15459c75cac8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:04.466442 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5de06-e649-4b26-a009-15459c75cac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:04.466719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5de06-e649-4b26-a009-15459c75cac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:04.482803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5de06-e649-4b26-a009-15459c75cac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:04.484292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5de06-e649-4b26-a009-15459c75cac8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:04.499456 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:04.505821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-fcf5de06-e649-4b26-a009-15459c75cac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:04.505821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:04.505821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:04.505821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:04.505821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:04.505821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:04.505821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:04.505821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:04.505821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:04.505821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:04.505821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:04.505821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:04.505821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:04.505821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:04.505821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:04.505821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:04.506899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:04.506899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:04.506899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:04.506899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:04.506899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:04.506899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:04.506899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:04.506899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:04.506899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:04.506899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:04.506899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:04.506899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:04.506899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:04.506899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:04.506899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:04.506899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:04.506899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:04.507809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:04.507809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:04.507809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:04.512251 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 426/858] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:04.513310 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5de06-e649-4b26-a009-15459c75cac8 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:04.581711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 433/859] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:31:04 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:31:04.587966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72ae49-a659-4915-a0d0-ed2226d06516 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:04.668731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5de06-e649-4b26-a009-15459c75cac8 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:04.715459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249fd26-477e-4fde-9fd8-fe8080405c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:04.716576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249fd26-477e-4fde-9fd8-fe8080405c28 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:04.717073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249fd26-477e-4fde-9fd8-fe8080405c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:04.717594 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249fd26-477e-4fde-9fd8-fe8080405c28 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:04.718471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249fd26-477e-4fde-9fd8-fe8080405c28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:04.719084 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249fd26-477e-4fde-9fd8-fe8080405c28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:04.719267 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72ae49-a659-4915-a0d0-ed2226d06516 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:04.721058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249fd26-477e-4fde-9fd8-fe8080405c28 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:04.736874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 427/860] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:04.742926 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249fd26-477e-4fde-9fd8-fe8080405c28 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:04.768667 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41174070-ac91-4f7f-8489-5e4e568c10bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:04.769106 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41174070-ac91-4f7f-8489-5e4e568c10bb None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:04.769404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41174070-ac91-4f7f-8489-5e4e568c10bb None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:04.769669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41174070-ac91-4f7f-8489-5e4e568c10bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:04.782999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41174070-ac91-4f7f-8489-5e4e568c10bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:04.784520 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41174070-ac91-4f7f-8489-5e4e568c10bb None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:04.796505 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:04.797315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:04.807381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-41174070-ac91-4f7f-8489-5e4e568c10bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:04.807381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:04.807381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:04.807381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:04.807381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:04.807381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:04.807381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:04.807381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:04.807381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:04.807381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:04.807381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:04.807381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:04.807381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:04.807381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:04.807381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:04.807381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:04.808678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:04.808678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:04.808678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:04.808678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:04.808678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:04.808678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:04.808678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:04.808678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:04.808678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:04.808678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:04.808678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:04.808678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:04.808678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:04.808678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:04.808678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:04.808678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:04.808678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:04.809843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:04.809843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:04.809843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:04.816456 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 434/861] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:04.817468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41174070-ac91-4f7f-8489-5e4e568c10bb None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:04.904738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249fd26-477e-4fde-9fd8-fe8080405c28 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:04.948219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41174070-ac91-4f7f-8489-5e4e568c10bb None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:04.953447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa208295-e43c-4e72-9e8d-ada45b461801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:04.954720 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa208295-e43c-4e72-9e8d-ada45b461801 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:04.955145 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa208295-e43c-4e72-9e8d-ada45b461801 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:04.955584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa208295-e43c-4e72-9e8d-ada45b461801 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:04.966279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa208295-e43c-4e72-9e8d-ada45b461801 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:04.967953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa208295-e43c-4e72-9e8d-ada45b461801 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:04.980905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-fa208295-e43c-4e72-9e8d-ada45b461801 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:04.980905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:04.980905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:04.980905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:04.980905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:04.980905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:04.980905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:04.980905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:04.980905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:04.980905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:04.980905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:04.980905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:04.980905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:31:04.980905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:31:04.980905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:31:04.982132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:31:04.982132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:04.982132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:04.982132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:04.982132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:04.982132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:04.982132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:04.982132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:04.982132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:04.982132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:04.982132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:04.982132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:04.982132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:04.982132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:04.982132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:04.982132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:04.982132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:04.983048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:31:04.983048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:31:04.983048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:31:04.983048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:31:04.983048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:31:04.983048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:31:04.983048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:04.983048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:04.988317 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 428/862] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:31:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:04.989420 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa208295-e43c-4e72-9e8d-ada45b461801 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:04.995493 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556f190-2d24-4e32-bf30-b0231e66a855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:04.996471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556f190-2d24-4e32-bf30-b0231e66a855 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:04.996935 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556f190-2d24-4e32-bf30-b0231e66a855 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:04.997415 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556f190-2d24-4e32-bf30-b0231e66a855 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:04.998271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556f190-2d24-4e32-bf30-b0231e66a855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:04.998784 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556f190-2d24-4e32-bf30-b0231e66a855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:05.000370 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556f190-2d24-4e32-bf30-b0231e66a855 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:05.019974 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 435/863] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:05.021769 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556f190-2d24-4e32-bf30-b0231e66a855 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:05.089694 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:05.095306 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:05.106763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa208295-e43c-4e72-9e8d-ada45b461801 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:05.144493 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05d84f-e699-49e9-9fc4-af5c278d42b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:05.145331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05d84f-e699-49e9-9fc4-af5c278d42b0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:05.145710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05d84f-e699-49e9-9fc4-af5c278d42b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:05.146090 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05d84f-e699-49e9-9fc4-af5c278d42b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:05.146892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05d84f-e699-49e9-9fc4-af5c278d42b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:05.147525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05d84f-e699-49e9-9fc4-af5c278d42b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:05.148853 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05d84f-e699-49e9-9fc4-af5c278d42b0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:05.150747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556f190-2d24-4e32-bf30-b0231e66a855 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:05.168921 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 429/864] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:31:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:05.170274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05d84f-e699-49e9-9fc4-af5c278d42b0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:05.193445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a014e-69fb-49ac-a2b7-1bfb4f7f9350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:05.194368 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a014e-69fb-49ac-a2b7-1bfb4f7f9350 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:05.194799 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a014e-69fb-49ac-a2b7-1bfb4f7f9350 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:05.195241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a014e-69fb-49ac-a2b7-1bfb4f7f9350 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:05.205941 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a014e-69fb-49ac-a2b7-1bfb4f7f9350 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:05.208261 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a014e-69fb-49ac-a2b7-1bfb4f7f9350 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:05.221811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-f12a014e-69fb-49ac-a2b7-1bfb4f7f9350 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:05.221811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:05.221811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:05.221811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:05.221811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:05.221811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:05.221811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:05.221811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:05.221811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:05.221811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:05.221811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:05.221811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:05.221811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:31:05.221811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:31:05.221811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:31:05.222817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:31:05.222817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:05.222817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:05.222817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:05.222817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:05.222817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:05.222817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:05.222817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:05.222817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:05.222817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:05.222817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:05.222817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:05.222817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:05.222817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:05.222817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:05.222817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:05.222817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:05.223715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:31:05.223715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:31:05.223715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:31:05.223715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:31:05.223715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:31:05.223715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:31:05.223715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:05.223715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:05.229972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 436/865] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:31:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:05.233212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a014e-69fb-49ac-a2b7-1bfb4f7f9350 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:05.342093 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05d84f-e699-49e9-9fc4-af5c278d42b0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:05.356571 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a014e-69fb-49ac-a2b7-1bfb4f7f9350 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:05.382013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:05.389489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706758b0-05bf-4624-afa1-da591cb53202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:05.390683 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706758b0-05bf-4624-afa1-da591cb53202 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:05.391144 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706758b0-05bf-4624-afa1-da591cb53202 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:05.391580 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706758b0-05bf-4624-afa1-da591cb53202 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:05.392683 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706758b0-05bf-4624-afa1-da591cb53202 None admin] RBAC: Adding query filter params (role.id=881210e567974dc0aa3043393c00a838, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=0216bf67389e4627bf953b7dcc97a88c) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:05.393317 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706758b0-05bf-4624-afa1-da591cb53202 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:05.395405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706758b0-05bf-4624-afa1-da591cb53202 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:05.401658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6ffbc-7be2-4fed-b7b8-bf009bc92fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:05.402831 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6ffbc-7be2-4fed-b7b8-bf009bc92fff None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:05.403266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6ffbc-7be2-4fed-b7b8-bf009bc92fff None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:05.403689 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6ffbc-7be2-4fed-b7b8-bf009bc92fff None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:05.404558 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6ffbc-7be2-4fed-b7b8-bf009bc92fff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:05.405031 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6ffbc-7be2-4fed-b7b8-bf009bc92fff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:05.406692 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6ffbc-7be2-4fed-b7b8-bf009bc92fff None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:05.411466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 430/866] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:31:05 2022] GET /identity/v3/role_assignments?user.id=0216bf67389e4627bf953b7dcc97a88c&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=881210e567974dc0aa3043393c00a838 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:05.423409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 437/867] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:31:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:05.430084 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706758b0-05bf-4624-afa1-da591cb53202 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:05.449252 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:05.563162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706758b0-05bf-4624-afa1-da591cb53202 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:05.607413 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d6bb5-a301-4653-9f49-b0ad1fe516f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:05.608062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d6bb5-a301-4653-9f49-b0ad1fe516f1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:05.608341 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d6bb5-a301-4653-9f49-b0ad1fe516f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:05.608612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d6bb5-a301-4653-9f49-b0ad1fe516f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:05.609433 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d6bb5-a301-4653-9f49-b0ad1fe516f1 None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=00f258aba84d4dd59ebe8de973ac65c6) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:05.609746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d6bb5-a301-4653-9f49-b0ad1fe516f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:05.611314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d6bb5-a301-4653-9f49-b0ad1fe516f1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:05.619155 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 431/868] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:31:05 2022] GET /identity/v3/role_assignments?user.id=00f258aba84d4dd59ebe8de973ac65c6&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:05.752005 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8d37f-f707-47e5-9900-02dd3fc11168 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:05.752336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8d37f-f707-47e5-9900-02dd3fc11168 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:05.752653 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8d37f-f707-47e5-9900-02dd3fc11168 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:05.754315 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 438/869] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:05.760482 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc6a2b-c218-4151-be36-2427c1deb68c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:05.760944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc6a2b-c218-4151-be36-2427c1deb68c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:05.761124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc6a2b-c218-4151-be36-2427c1deb68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:05.783929 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-45dc6a2b-c218-4151-be36-2427c1deb68c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:05.785281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-45dc6a2b-c218-4151-be36-2427c1deb68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:05.886019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 432/870] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:05 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:06.076016 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90448ea1-9850-47dd-82ba-89ee43193921 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:06.076409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90448ea1-9850-47dd-82ba-89ee43193921 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:06.076572 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90448ea1-9850-47dd-82ba-89ee43193921 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:06.078243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 439/871] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:06.084058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0293845-e0e5-41bc-88be-3d0332a684a1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:06.084604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0293845-e0e5-41bc-88be-3d0332a684a1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:06.084604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0293845-e0e5-41bc-88be-3d0332a684a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:06.102861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:06.112595 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f0293845-e0e5-41bc-88be-3d0332a684a1 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:06.114502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f0293845-e0e5-41bc-88be-3d0332a684a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:06.176201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90448ea1-9850-47dd-82ba-89ee43193921 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:06.261214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 433/872] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:06 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:06.346774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90448ea1-9850-47dd-82ba-89ee43193921 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:06.412137 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b090a9a-bc20-4476-8b88-957fcca2461a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:06.413233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b090a9a-bc20-4476-8b88-957fcca2461a None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:06.413729 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b090a9a-bc20-4476-8b88-957fcca2461a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:06.414271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b090a9a-bc20-4476-8b88-957fcca2461a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:06.415324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b090a9a-bc20-4476-8b88-957fcca2461a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:06.418079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b090a9a-bc20-4476-8b88-957fcca2461a None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:06.435160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-0b090a9a-bc20-4476-8b88-957fcca2461a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:31:06.435160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:06.435160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:31:06.435160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:31:06.435160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:31:06.435160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:31:06.435160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:31:06.435160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:06.435160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:31:06.435160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:06.435160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:06.435160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:06.435160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:06.435160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:06.435160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:06.435160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:06.436194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:06.436194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:06.436194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:06.436194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:06.436194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:06.436194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:06.436194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:06.436194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:31:06.436194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:31:06.436194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:31:06.436194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:31:06.436194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:06.436194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:06.436194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:06.436194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:06.436194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:06.436194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:06.437156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:06.437156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:06.437156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:06.437156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:06.437156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:06.437156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:06.437156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:06.437156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:06.437156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:06.437156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:06.437156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:31:06.437156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:06.437156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:31:06.437156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:06.438010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 440/873] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Sep 26 05:31:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:06.445273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0293845-e0e5-41bc-88be-3d0332a684a1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:06.540073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b090a9a-bc20-4476-8b88-957fcca2461a None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:06.598834 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0293845-e0e5-41bc-88be-3d0332a684a1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:06.641255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfe168-1664-4a71-9100-e867f97fba6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:06.642382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfe168-1664-4a71-9100-e867f97fba6f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:06.642871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfe168-1664-4a71-9100-e867f97fba6f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:06.643386 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfe168-1664-4a71-9100-e867f97fba6f None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:06.644278 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfe168-1664-4a71-9100-e867f97fba6f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:06.644796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfe168-1664-4a71-9100-e867f97fba6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:06.646408 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:06.647213 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfe168-1664-4a71-9100-e867f97fba6f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:06.653507 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:06.664506 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 434/874] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 26 05:31:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:06.665942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfe168-1664-4a71-9100-e867f97fba6f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:06.742921 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b090a9a-bc20-4476-8b88-957fcca2461a None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:06.785044 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8e545-5a8c-461e-a383-d69845c3e580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:06.786158 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8e545-5a8c-461e-a383-d69845c3e580 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:06.786653 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8e545-5a8c-461e-a383-d69845c3e580 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:06.787242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8e545-5a8c-461e-a383-d69845c3e580 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:06.788042 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8e545-5a8c-461e-a383-d69845c3e580 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:06.789775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8e545-5a8c-461e-a383-d69845c3e580 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:06.805287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-7ac8e545-5a8c-461e-a383-d69845c3e580 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:31:06.805287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:06.805287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:31:06.805287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:31:06.805287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:31:06.805287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:31:06.805287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:31:06.805287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:06.805287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:31:06.805287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:06.805287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:06.805287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:06.805287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:06.805287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:06.805287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:06.805287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:06.805287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:06.806378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:06.806378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:06.806378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:06.806378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:06.806378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:06.806378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:06.806378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:31:06.806378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:31:06.806378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:31:06.806378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:31:06.806378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:06.806378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:06.806378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:06.806378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:06.806378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:06.806378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:06.807224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:06.807224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:06.807224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:06.807224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:06.807224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:06.807224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:06.807224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:06.807224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:06.807224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:06.807224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:06.807224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:31:06.807224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:06.807224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:31:06.807224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:06.815086 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 441/875] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:31:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:06.816409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8e545-5a8c-461e-a383-d69845c3e580 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:06.856218 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfe168-1664-4a71-9100-e867f97fba6f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:06.937690 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d591bcb-c0a6-409e-9248-e9c67b703f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:06.939232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d591bcb-c0a6-409e-9248-e9c67b703f60 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:06.940056 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d591bcb-c0a6-409e-9248-e9c67b703f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:06.940830 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d591bcb-c0a6-409e-9248-e9c67b703f60 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:06.942341 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d591bcb-c0a6-409e-9248-e9c67b703f60 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:06.942964 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d591bcb-c0a6-409e-9248-e9c67b703f60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:06.945801 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d591bcb-c0a6-409e-9248-e9c67b703f60 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:06.961754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 435/876] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 26 05:31:06 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:06.987532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8e545-5a8c-461e-a383-d69845c3e580 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:07.008348 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:07.058894 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443b724-0774-44d5-a1f7-5747c4596e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:07.059905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443b724-0774-44d5-a1f7-5747c4596e23 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:07.060353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443b724-0774-44d5-a1f7-5747c4596e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:07.060912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443b724-0774-44d5-a1f7-5747c4596e23 None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:07.062071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443b724-0774-44d5-a1f7-5747c4596e23 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:07.062558 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443b724-0774-44d5-a1f7-5747c4596e23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:07.065113 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443b724-0774-44d5-a1f7-5747c4596e23 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:07.074561 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 442/877] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:31:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:07.082841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d591bcb-c0a6-409e-9248-e9c67b703f60 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:07.163163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:07.217227 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d591bcb-c0a6-409e-9248-e9c67b703f60 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:07.258459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb0844-2491-4fc4-b320-3ef0e9a5622f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:07.258973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb0844-2491-4fc4-b320-3ef0e9a5622f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:07.259246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb0844-2491-4fc4-b320-3ef0e9a5622f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:07.259513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb0844-2491-4fc4-b320-3ef0e9a5622f None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:07.260148 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb0844-2491-4fc4-b320-3ef0e9a5622f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:07.260469 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb0844-2491-4fc4-b320-3ef0e9a5622f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:07.261997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb0844-2491-4fc4-b320-3ef0e9a5622f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:07.270257 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 436/878] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:31:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:07.275500 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443b724-0774-44d5-a1f7-5747c4596e23 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:07.403506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443b724-0774-44d5-a1f7-5747c4596e23 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:07.442967 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eec49b-68ee-4d5a-b998-ff6cd29ff681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:07.443959 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eec49b-68ee-4d5a-b998-ff6cd29ff681 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:07.444470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eec49b-68ee-4d5a-b998-ff6cd29ff681 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:07.444935 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eec49b-68ee-4d5a-b998-ff6cd29ff681 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:07.445880 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eec49b-68ee-4d5a-b998-ff6cd29ff681 None admin] RBAC: Adding query filter params (interface=public, service_id=34f9c70a14404a69ac646c7831ca8fb7, region_id=RegionOne) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:07.446389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eec49b-68ee-4d5a-b998-ff6cd29ff681 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:07.447940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eec49b-68ee-4d5a-b998-ff6cd29ff681 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:07.454719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 443/879] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Sep 26 05:31:07 2022] GET /identity/v3/endpoints?service_id=34f9c70a14404a69ac646c7831ca8fb7&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:07.461852 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb0844-2491-4fc4-b320-3ef0e9a5622f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:07.586985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb0844-2491-4fc4-b320-3ef0e9a5622f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:07.633176 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:07.647802 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309913fc-cfb2-443e-af09-e504edc7dca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:07.648798 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309913fc-cfb2-443e-af09-e504edc7dca3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:07.649322 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309913fc-cfb2-443e-af09-e504edc7dca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:07.649808 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309913fc-cfb2-443e-af09-e504edc7dca3 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:07.650896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309913fc-cfb2-443e-af09-e504edc7dca3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:07.653538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309913fc-cfb2-443e-af09-e504edc7dca3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:07.664736 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 437/880] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 26 05:31:07 2022] GET /identity/v3/services => generated 1011 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:31:08.734951 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ca38a-feee-4ff4-ac8d-89853a0f7dbf None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:08.735992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ca38a-feee-4ff4-ac8d-89853a0f7dbf None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:08.736475 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ca38a-feee-4ff4-ac8d-89853a0f7dbf None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:08.738362 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 444/881] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:08.748339 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceec3b8-0ea1-4ec0-9be8-5e6788ff9a8d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:08.748879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceec3b8-0ea1-4ec0-9be8-5e6788ff9a8d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:08.749341 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceec3b8-0ea1-4ec0-9be8-5e6788ff9a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:08.780797 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-bceec3b8-0ea1-4ec0-9be8-5e6788ff9a8d None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:08.782793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bceec3b8-0ea1-4ec0-9be8-5e6788ff9a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:08.921653 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 438/882] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:08 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:09.189771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ca38a-feee-4ff4-ac8d-89853a0f7dbf None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:09.237112 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:09.303002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ca38a-feee-4ff4-ac8d-89853a0f7dbf None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:09.341546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42101f60-1e09-4c6d-b64f-89d71f96ca01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:09.342202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42101f60-1e09-4c6d-b64f-89d71f96ca01 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:09.342477 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42101f60-1e09-4c6d-b64f-89d71f96ca01 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:09.342742 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42101f60-1e09-4c6d-b64f-89d71f96ca01 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:09.348837 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace57da-5a28-4972-b658-f78f3eb08c18 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:09.349965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace57da-5a28-4972-b658-f78f3eb08c18 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:09.350774 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace57da-5a28-4972-b658-f78f3eb08c18 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:09.360156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42101f60-1e09-4c6d-b64f-89d71f96ca01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:31:09.360716 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 439/883] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:09.361114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-42101f60-1e09-4c6d-b64f-89d71f96ca01 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:31:09.361114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:09.361114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:09.361114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:09.361114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:09.361114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:09.361114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:09.361114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:09.361114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:09.361114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:09.361114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:09.361114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:09.361114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:31:09.361114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:31:09.361114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:31:09.361114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:31:09.363221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:31:09.363221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:31:09.363221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:31:09.363221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:31:09.363221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:09.363221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:09.363221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:09.363221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:09.363221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:09.363221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:09.363221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:09.363221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:09.363221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:09.363221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:09.363221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:09.363221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:09.365198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:09.365198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:09.365198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:09.365198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:09.365198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:31:09.365198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:31:09.365198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:31:09.365198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:31:09.365198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:31:09.365198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:31:09.365198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:31:09.365198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:09.365198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 445/884] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Sep 26 05:31:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:09.366547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e6176-d3af-4b30-b51d-185ea7bcc877 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:09.366547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e6176-d3af-4b30-b51d-185ea7bcc877 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:09.366547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e6176-d3af-4b30-b51d-185ea7bcc877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:09.370776 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42101f60-1e09-4c6d-b64f-89d71f96ca01 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:09.401539 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-0c7e6176-d3af-4b30-b51d-185ea7bcc877 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:09.404211 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0c7e6176-d3af-4b30-b51d-185ea7bcc877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:09.485770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:09.494122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42101f60-1e09-4c6d-b64f-89d71f96ca01 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:09.549171 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 440/885] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:09 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:31:09.553727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97d0dc-371e-4112-aea0-504017e33c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:09.554367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97d0dc-371e-4112-aea0-504017e33c32 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:09.554645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97d0dc-371e-4112-aea0-504017e33c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:09.554908 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97d0dc-371e-4112-aea0-504017e33c32 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:09.555617 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97d0dc-371e-4112-aea0-504017e33c32 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:09.561174 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97d0dc-371e-4112-aea0-504017e33c32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:09.561174 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97d0dc-371e-4112-aea0-504017e33c32 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:09.568155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 446/886] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Sep 26 05:31:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:09.574537 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e6176-d3af-4b30-b51d-185ea7bcc877 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:09.642559 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:09.702622 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e6176-d3af-4b30-b51d-185ea7bcc877 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:09.744800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d706d-eb42-486b-b80d-740ff225b0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:09.745530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d706d-eb42-486b-b80d-740ff225b0d6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:09.745827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d706d-eb42-486b-b80d-740ff225b0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:09.746067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d706d-eb42-486b-b80d-740ff225b0d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:09.758776 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d706d-eb42-486b-b80d-740ff225b0d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:09.760301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d706d-eb42-486b-b80d-740ff225b0d6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:09.778258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-9d5d706d-eb42-486b-b80d-740ff225b0d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:09.778258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:09.778258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:09.778258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:09.778258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:09.778258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:09.778258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:09.778258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:09.778258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:09.778258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:09.778258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:09.778258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:09.778258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:09.778258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:09.778258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:09.778258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:09.780376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:09.780376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:09.780376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:09.780376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:09.780376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:09.780376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:09.780376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:09.780376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:09.780376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:09.780376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:09.780376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:09.780376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:09.780376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:09.780376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:09.780376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:09.780376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:09.780376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:09.782168 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:09.782168 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:09.782168 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:09.782168 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 441/887] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:09.786042 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97d0dc-371e-4112-aea0-504017e33c32 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:09.917177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97d0dc-371e-4112-aea0-504017e33c32 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:09.950276 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812cc85-ded3-443b-9c4f-ad0a8c5ee2df None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:09.950709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812cc85-ded3-443b-9c4f-ad0a8c5ee2df None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:09.951032 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812cc85-ded3-443b-9c4f-ad0a8c5ee2df None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:09.952450 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 442/888] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:09.953632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e4af0-9c45-4c22-8094-27b9fdaa4669 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:09.953913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e4af0-9c45-4c22-8094-27b9fdaa4669 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:09.954184 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e4af0-9c45-4c22-8094-27b9fdaa4669 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:09.955671 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 443/889] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:09.955890 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cc60c-2a9f-400d-b039-12cc79fdb99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:09.956546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cc60c-2a9f-400d-b039-12cc79fdb99f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:09.956825 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cc60c-2a9f-400d-b039-12cc79fdb99f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:09.959566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cc60c-2a9f-400d-b039-12cc79fdb99f None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:09.959566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283cc60c-2a9f-400d-b039-12cc79fdb99f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:09.959566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283cc60c-2a9f-400d-b039-12cc79fdb99f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:09.959566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283cc60c-2a9f-400d-b039-12cc79fdb99f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:09.961409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a8fde-9499-4edf-aaea-a88875e9325a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:09.961545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a8fde-9499-4edf-aaea-a88875e9325a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:09.961714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a8fde-9499-4edf-aaea-a88875e9325a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:09.974684 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 447/890] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:09.979771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0834c0e-75fb-407d-bc6e-769d3823bcc2 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:09.979771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0834c0e-75fb-407d-bc6e-769d3823bcc2 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:09.979771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0834c0e-75fb-407d-bc6e-769d3823bcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:09.981789 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-db7a8fde-9499-4edf-aaea-a88875e9325a None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:09.983011 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-db7a8fde-9499-4edf-aaea-a88875e9325a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:09.999356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-c0834c0e-75fb-407d-bc6e-769d3823bcc2 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:10.000688 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c0834c0e-75fb-407d-bc6e-769d3823bcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:10.016936 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:10.082322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 448/891] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:09 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:10.083195 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0834c0e-75fb-407d-bc6e-769d3823bcc2 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:10.099807 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 444/892] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:09 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:10.101815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a8fde-9499-4edf-aaea-a88875e9325a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:10.186651 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:10.213421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0834c0e-75fb-407d-bc6e-769d3823bcc2 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:10.288911 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f53d10-f7a2-4a4f-8b98-d28dadd43555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:10.289509 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f53d10-f7a2-4a4f-8b98-d28dadd43555 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:10.289772 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f53d10-f7a2-4a4f-8b98-d28dadd43555 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:10.290028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f53d10-f7a2-4a4f-8b98-d28dadd43555 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:10.299929 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f53d10-f7a2-4a4f-8b98-d28dadd43555 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:10.301542 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f53d10-f7a2-4a4f-8b98-d28dadd43555 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:10.313756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a8fde-9499-4edf-aaea-a88875e9325a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:10.317290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-99f53d10-f7a2-4a4f-8b98-d28dadd43555 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:31:10.317290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:10.317290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:10.317290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:10.317290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:10.317290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:10.317290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:10.317290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:10.317290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:10.317290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:10.317290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:10.317290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:10.317290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:31:10.317290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:31:10.317290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:31:10.317290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:10.319202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:10.320995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:10.320995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:10.320995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:31:10.320995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:31:10.320995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:31:10.320995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:31:10.320995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:31:10.320995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:31:10.320995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:31:10.320995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:10.320995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 449/893] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 26 05:31:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:10.328905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f53d10-f7a2-4a4f-8b98-d28dadd43555 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:10.355821 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:10.369091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507cbaa-7c2b-4133-b779-3f50994809c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:10.369772 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507cbaa-7c2b-4133-b779-3f50994809c3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:10.370449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507cbaa-7c2b-4133-b779-3f50994809c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:10.370747 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507cbaa-7c2b-4133-b779-3f50994809c3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:10.371338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b507cbaa-7c2b-4133-b779-3f50994809c3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:10.373702 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b507cbaa-7c2b-4133-b779-3f50994809c3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:10.385382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b507cbaa-7c2b-4133-b779-3f50994809c3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:31:10.385382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:10.385382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:31:10.385382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:31:10.385382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:31:10.385382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:31:10.385382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:31:10.385382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:10.385382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:31:10.385382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:10.385382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:10.385382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:10.385382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:10.385382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:10.385382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:10.385382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:10.385382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:10.387345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:10.387345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:10.387345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:10.387345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:10.387345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:10.387345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:10.387345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:31:10.387345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:31:10.387345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:31:10.387345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:31:10.387345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:10.387345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:10.387345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:10.387345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:10.387345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:10.387345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:10.388823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:10.388823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:10.388823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:10.388823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:10.388823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:10.388823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:10.388823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:10.388823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:10.388823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:10.388823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:10.388823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:31:10.388823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:10.388823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:31:10.388823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:10.388823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 445/894] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 26 05:31:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:10.396445 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507cbaa-7c2b-4133-b779-3f50994809c3 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:10.494836 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f53d10-f7a2-4a4f-8b98-d28dadd43555 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:10.541974 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dce623-de52-4f1f-9d48-84f0a4d2a3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:10.542506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dce623-de52-4f1f-9d48-84f0a4d2a3cf None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:10.542779 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dce623-de52-4f1f-9d48-84f0a4d2a3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:10.543053 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dce623-de52-4f1f-9d48-84f0a4d2a3cf None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:10.543788 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dce623-de52-4f1f-9d48-84f0a4d2a3cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:10.544092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dce623-de52-4f1f-9d48-84f0a4d2a3cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:10.545867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dce623-de52-4f1f-9d48-84f0a4d2a3cf None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:10.570955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 450/895] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Sep 26 05:31:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:10.571905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dce623-de52-4f1f-9d48-84f0a4d2a3cf None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:10.611759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:10.618287 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507cbaa-7c2b-4133-b779-3f50994809c3 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:10.671557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac7d43-4374-4192-9323-f16f5b06702c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:10.672496 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac7d43-4374-4192-9323-f16f5b06702c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:10.672938 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac7d43-4374-4192-9323-f16f5b06702c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:10.673406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac7d43-4374-4192-9323-f16f5b06702c None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:10.674030 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:10.674253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac7d43-4374-4192-9323-f16f5b06702c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:10.674757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac7d43-4374-4192-9323-f16f5b06702c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:10.677432 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac7d43-4374-4192-9323-f16f5b06702c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:10.689709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 446/896] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 26 05:31:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:10.689709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac7d43-4374-4192-9323-f16f5b06702c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:10.730718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dce623-de52-4f1f-9d48-84f0a4d2a3cf None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:10.787032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12b272-32a4-463a-aaee-7f34cb4215af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:10.788086 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12b272-32a4-463a-aaee-7f34cb4215af None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:10.788584 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12b272-32a4-463a-aaee-7f34cb4215af None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:10.789047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12b272-32a4-463a-aaee-7f34cb4215af None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:10.790198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12b272-32a4-463a-aaee-7f34cb4215af None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:10.793047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12b272-32a4-463a-aaee-7f34cb4215af None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:10.809969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-9f12b272-32a4-463a-aaee-7f34cb4215af None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:31:10.809969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:10.809969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:31:10.809969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:31:10.809969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:31:10.809969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:31:10.809969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:31:10.809969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:10.809969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:31:10.809969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:10.809969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:10.809969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:10.809969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:10.809969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:10.809969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:10.809969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:10.809969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:10.811353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:10.811353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:10.811353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:10.811353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:10.811353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:10.811353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:10.811353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:31:10.811353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:31:10.811353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:31:10.811353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:31:10.811353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:10.811353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:10.811353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:10.811353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:10.811353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:10.811353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:10.812796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:10.812796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:10.812796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:10.812796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:10.812796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:10.812796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:10.812796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:10.812796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:10.812796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:10.812796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:10.812796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:31:10.812796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:10.812796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 05:31:10.812796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:10.812796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 451/897] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:31:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:10.814356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12b272-32a4-463a-aaee-7f34cb4215af None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:10.842356 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac7d43-4374-4192-9323-f16f5b06702c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:10.885101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9738626-8538-4099-a495-eef467906737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:10.886191 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9738626-8538-4099-a495-eef467906737 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:10.886627 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9738626-8538-4099-a495-eef467906737 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:10.887058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9738626-8538-4099-a495-eef467906737 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:10.928236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:10.947255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12b272-32a4-463a-aaee-7f34cb4215af None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:10.981524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9738626-8538-4099-a495-eef467906737 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:10.983306 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9738626-8538-4099-a495-eef467906737 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:10.991352 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:10.999418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8314986-92d5-4601-a106-8aa48bf49c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:11.000168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8314986-92d5-4601-a106-8aa48bf49c48 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:11.000551 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8314986-92d5-4601-a106-8aa48bf49c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:11.000901 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8314986-92d5-4601-a106-8aa48bf49c48 None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:11.001826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8314986-92d5-4601-a106-8aa48bf49c48 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:11.003420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8314986-92d5-4601-a106-8aa48bf49c48 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:11.021905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 452/898] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 26 05:31:10 2022] POST /identity/v3/services => generated 269 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:31:11.023028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8314986-92d5-4601-a106-8aa48bf49c48 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:11.188634 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 447/899] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:31:10 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:31:11.188634 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9738626-8538-4099-a495-eef467906737 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:11.225679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8314986-92d5-4601-a106-8aa48bf49c48 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:11.296049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19db12-32df-411c-a2b7-323c4f23647e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:11.296959 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19db12-32df-411c-a2b7-323c4f23647e None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:11.297344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19db12-32df-411c-a2b7-323c4f23647e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:11.297789 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19db12-32df-411c-a2b7-323c4f23647e None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:11.298823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19db12-32df-411c-a2b7-323c4f23647e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:11.299320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19db12-32df-411c-a2b7-323c4f23647e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:11.301538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19db12-32df-411c-a2b7-323c4f23647e None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:11.308771 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:11.309464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 453/900] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 26 05:31:11 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:11.310861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19db12-32df-411c-a2b7-323c4f23647e None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:11.400291 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9738626-8538-4099-a495-eef467906737 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:11.429122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:11.492811 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac06226-8015-4165-b670-6a8c86fab30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:11.495600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac06226-8015-4165-b670-6a8c86fab30d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:11.496633 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac06226-8015-4165-b670-6a8c86fab30d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:11.503061 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac06226-8015-4165-b670-6a8c86fab30d None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:11.505165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac06226-8015-4165-b670-6a8c86fab30d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:11.507765 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac06226-8015-4165-b670-6a8c86fab30d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:11.512495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac06226-8015-4165-b670-6a8c86fab30d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:11.532319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 448/901] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:31:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:11.541540 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac06226-8015-4165-b670-6a8c86fab30d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:11.589028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19db12-32df-411c-a2b7-323c4f23647e None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:11.695327 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68030d77-5eef-407f-8a65-ac43f1f6ca9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:11.696499 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68030d77-5eef-407f-8a65-ac43f1f6ca9c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:11.696974 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68030d77-5eef-407f-8a65-ac43f1f6ca9c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:11.697499 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68030d77-5eef-407f-8a65-ac43f1f6ca9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:11.723169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68030d77-5eef-407f-8a65-ac43f1f6ca9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:11.726704 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68030d77-5eef-407f-8a65-ac43f1f6ca9c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:11.757778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-68030d77-5eef-407f-8a65-ac43f1f6ca9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:11.757778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:11.757778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:11.757778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:11.757778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:11.757778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:11.757778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:11.757778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:11.757778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:11.757778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:11.757778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:11.757778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:11.757778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:11.757778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:11.757778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:11.757778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:11.759510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:11.759510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:11.759510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:11.759510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:11.759510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:11.759510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:11.759510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:11.759510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:11.759510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:11.759510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:11.759510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:11.759510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:11.759510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:11.759510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:11.759510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:11.759510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:11.759510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:11.769722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:11.769722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:11.769722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:11.769722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 454/902] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 450 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:11.773666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68030d77-5eef-407f-8a65-ac43f1f6ca9c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:11.824084 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac06226-8015-4165-b670-6a8c86fab30d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:11.913720 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:11.946879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a9060-23d1-4799-88a7-410be8126f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:11.947512 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a9060-23d1-4799-88a7-410be8126f49 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:11.947789 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a9060-23d1-4799-88a7-410be8126f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:11.948068 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a9060-23d1-4799-88a7-410be8126f49 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:11.948781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a9060-23d1-4799-88a7-410be8126f49 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:11.955942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a9060-23d1-4799-88a7-410be8126f49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:11.958992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a9060-23d1-4799-88a7-410be8126f49 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:11.985855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 449/903] 10.222.0.57 () {56 vars in 1114 bytes} [Mon Sep 26 05:31:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:11.986885 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a9060-23d1-4799-88a7-410be8126f49 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:12.040450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68030d77-5eef-407f-8a65-ac43f1f6ca9c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:12.051888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:12.133898 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994db70-3087-4e3b-9ba8-cfeafcda43ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:12.135937 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994db70-3087-4e3b-9ba8-cfeafcda43ba None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:12.136716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994db70-3087-4e3b-9ba8-cfeafcda43ba None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:12.138189 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994db70-3087-4e3b-9ba8-cfeafcda43ba None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:12.139837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994db70-3087-4e3b-9ba8-cfeafcda43ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:12.140666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994db70-3087-4e3b-9ba8-cfeafcda43ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:12.143558 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994db70-3087-4e3b-9ba8-cfeafcda43ba None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:12.178931 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 455/904] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:12.180991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994db70-3087-4e3b-9ba8-cfeafcda43ba None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:12.267672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a9060-23d1-4799-88a7-410be8126f49 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:12.347916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97fa941-ac31-4cfb-b2f3-47ab038994d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:12.348660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fa941-ac31-4cfb-b2f3-47ab038994d7 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:12.348840 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fa941-ac31-4cfb-b2f3-47ab038994d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:12.349063 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fa941-ac31-4cfb-b2f3-47ab038994d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:12.349806 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97fa941-ac31-4cfb-b2f3-47ab038994d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:12.351256 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97fa941-ac31-4cfb-b2f3-47ab038994d7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:12.384606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 450/905] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:31:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:31:12.432648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994db70-3087-4e3b-9ba8-cfeafcda43ba None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:12.512755 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0aa58-b13a-44ef-9c8c-860c26e8dfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:12.514401 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0aa58-b13a-44ef-9c8c-860c26e8dfad None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:12.515151 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0aa58-b13a-44ef-9c8c-860c26e8dfad None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:12.515892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0aa58-b13a-44ef-9c8c-860c26e8dfad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:12.530486 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0aa58-b13a-44ef-9c8c-860c26e8dfad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:12.533902 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0aa58-b13a-44ef-9c8c-860c26e8dfad None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:12.557428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-0bc0aa58-b13a-44ef-9c8c-860c26e8dfad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:12.557428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:12.557428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:12.557428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:12.557428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:12.557428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:12.557428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:12.557428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:12.557428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:12.557428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:12.557428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:12.557428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:12.557428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:31:12.557428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:31:12.557428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:31:12.559305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:31:12.559305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:12.559305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:12.559305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:12.559305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:12.559305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:12.559305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:12.559305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:12.559305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:12.559305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:12.559305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:12.559305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:12.559305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:12.559305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:12.559305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:12.559305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:12.559305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:12.561007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:31:12.561007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:31:12.561007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:31:12.561007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:31:12.561007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:31:12.561007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:31:12.561007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:12.561007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:12.565545 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 456/906] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:31:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:12.575221 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97fa941-ac31-4cfb-b2f3-47ab038994d7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:12.654119 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:12.729519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97fa941-ac31-4cfb-b2f3-47ab038994d7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:12.772778 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1ac41-4be3-4fda-84ff-ebc58279e0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:12.773448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1ac41-4be3-4fda-84ff-ebc58279e0ba None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:12.773721 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1ac41-4be3-4fda-84ff-ebc58279e0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:12.773992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1ac41-4be3-4fda-84ff-ebc58279e0ba None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:12.774705 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1ac41-4be3-4fda-84ff-ebc58279e0ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:12.775011 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1ac41-4be3-4fda-84ff-ebc58279e0ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:12.776577 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1ac41-4be3-4fda-84ff-ebc58279e0ba None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:12.791471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 451/907] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:31:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:12.801698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0aa58-b13a-44ef-9c8c-860c26e8dfad None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:12.837881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:12.939979 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0aa58-b13a-44ef-9c8c-860c26e8dfad None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:12.978367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c5b5d-24bb-4270-b910-c83feaa7ddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:12.978996 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c5b5d-24bb-4270-b910-c83feaa7ddfe None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:12.979278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c5b5d-24bb-4270-b910-c83feaa7ddfe None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:12.979667 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c5b5d-24bb-4270-b910-c83feaa7ddfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:12.980362 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c5b5d-24bb-4270-b910-c83feaa7ddfe None admin] RBAC: Adding query filter params (role.id=aedef758940042499c7698d83130db60, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=0216bf67389e4627bf953b7dcc97a88c) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:12.980671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c5b5d-24bb-4270-b910-c83feaa7ddfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:12.982295 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c5b5d-24bb-4270-b910-c83feaa7ddfe None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:12.989109 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 457/908] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:31:12 2022] GET /identity/v3/role_assignments?user.id=0216bf67389e4627bf953b7dcc97a88c&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=aedef758940042499c7698d83130db60 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:14.208178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afc8b4-5c2c-41ec-be97-1168e15a9f9b None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:14.208986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afc8b4-5c2c-41ec-be97-1168e15a9f9b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:14.208986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afc8b4-5c2c-41ec-be97-1168e15a9f9b None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:14.210680 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 452/909] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:14.219074 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8b0a8-83ea-4ed1-ac72-f60102b68d02 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:14.219854 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8b0a8-83ea-4ed1-ac72-f60102b68d02 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:14.220013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8b0a8-83ea-4ed1-ac72-f60102b68d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:14.262735 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-42e8b0a8-83ea-4ed1-ac72-f60102b68d02 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:14.265167 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-42e8b0a8-83ea-4ed1-ac72-f60102b68d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:14.399190 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 458/910] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:14.489415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace782bd-e1b1-4b60-b83d-96d06f67443b None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:14.489659 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace782bd-e1b1-4b60-b83d-96d06f67443b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:14.489825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace782bd-e1b1-4b60-b83d-96d06f67443b None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:14.491494 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 453/911] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:14.497405 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ce741-f67c-4bf0-b4e1-3883c07aa9c5 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:14.497787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ce741-f67c-4bf0-b4e1-3883c07aa9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:14.497970 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ce741-f67c-4bf0-b4e1-3883c07aa9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:14.518698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-ae2ce741-f67c-4bf0-b4e1-3883c07aa9c5 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:14.519915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ae2ce741-f67c-4bf0-b4e1-3883c07aa9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:14.530253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:14.628625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 459/912] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:31:14.884292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace782bd-e1b1-4b60-b83d-96d06f67443b None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:14.894396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ce741-f67c-4bf0-b4e1-3883c07aa9c5 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:15.009362 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ce741-f67c-4bf0-b4e1-3883c07aa9c5 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:15.050184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c0862-a48a-4a8e-9207-a199f3f788ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:15.050843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c0862-a48a-4a8e-9207-a199f3f788ae None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:15.051112 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c0862-a48a-4a8e-9207-a199f3f788ae None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:15.051392 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c0862-a48a-4a8e-9207-a199f3f788ae None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:15.051978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c0862-a48a-4a8e-9207-a199f3f788ae None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:15.053425 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c0862-a48a-4a8e-9207-a199f3f788ae None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:15.064065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-536c0862-a48a-4a8e-9207-a199f3f788ae None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:31:15.064065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:15.064065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:31:15.064065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:31:15.064065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:31:15.064065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:31:15.064065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:31:15.064065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:15.064065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:31:15.064065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:15.064065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:15.064065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:15.064065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:15.064065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:15.064065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:15.064065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:15.065619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:15.065619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:15.065619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:15.065619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:15.065619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:15.065619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:15.065619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:15.065619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:31:15.065619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:31:15.065619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:31:15.065619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:31:15.065619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:15.065619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:15.065619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:15.065619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:15.065619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:15.065619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:15.069235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:15.069235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:15.069235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:15.069235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:15.069235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:15.069235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:15.069235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:15.069235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:15.069235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:15.069235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:15.069235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:31:15.069235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:15.069235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:31:15.069235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:15.077437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 460/913] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Sep 26 05:31:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:15.077437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c0862-a48a-4a8e-9207-a199f3f788ae None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:15.084710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:15.119248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace782bd-e1b1-4b60-b83d-96d06f67443b None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:15.135148 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:15.192200 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccef8a4-66b0-417f-bc3b-c02ad0984bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:15.193366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccef8a4-66b0-417f-bc3b-c02ad0984bd9 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:15.193860 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccef8a4-66b0-417f-bc3b-c02ad0984bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:15.194338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccef8a4-66b0-417f-bc3b-c02ad0984bd9 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:15.195550 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccef8a4-66b0-417f-bc3b-c02ad0984bd9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:15.198365 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccef8a4-66b0-417f-bc3b-c02ad0984bd9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:15.199370 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c0862-a48a-4a8e-9207-a199f3f788ae None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:15.227547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 454/914] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 26 05:31:14 2022] POST /identity/v3/services => generated 241 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:31:15.248774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd68a0f-8f24-48ba-9a2a-2ab79015e5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:15.249471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd68a0f-8f24-48ba-9a2a-2ab79015e5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:15.253344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd68a0f-8f24-48ba-9a2a-2ab79015e5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:15.253344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd68a0f-8f24-48ba-9a2a-2ab79015e5c5 None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:15.253344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd68a0f-8f24-48ba-9a2a-2ab79015e5c5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:15.253344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd68a0f-8f24-48ba-9a2a-2ab79015e5c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:15.253344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd68a0f-8f24-48ba-9a2a-2ab79015e5c5 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:15.258030 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 461/915] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 26 05:31:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:15.264515 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccef8a4-66b0-417f-bc3b-c02ad0984bd9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:15.446226 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccef8a4-66b0-417f-bc3b-c02ad0984bd9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:15.514975 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a161290e-730c-4377-a1d0-97e396445a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:15.516041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161290e-730c-4377-a1d0-97e396445a00 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:15.516519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161290e-730c-4377-a1d0-97e396445a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:15.517004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a161290e-730c-4377-a1d0-97e396445a00 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:15.518382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a161290e-730c-4377-a1d0-97e396445a00 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:15.518920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a161290e-730c-4377-a1d0-97e396445a00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:15.521561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a161290e-730c-4377-a1d0-97e396445a00 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:15.531934 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 455/916] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 26 05:31:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:15.538384 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd68a0f-8f24-48ba-9a2a-2ab79015e5c5 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:15.648753 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:15.688058 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd68a0f-8f24-48ba-9a2a-2ab79015e5c5 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:15.726770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8071ec6-0ebb-475a-91ae-41661e0e1602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:15.727299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8071ec6-0ebb-475a-91ae-41661e0e1602 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:15.727578 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8071ec6-0ebb-475a-91ae-41661e0e1602 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:15.727849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8071ec6-0ebb-475a-91ae-41661e0e1602 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:15.728597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8071ec6-0ebb-475a-91ae-41661e0e1602 None admin] RBAC: Adding query filter params (interface=public, service_id=5df2988ab57e491db664382ef5331005, region_id=RegionOne) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:15.728904 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8071ec6-0ebb-475a-91ae-41661e0e1602 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:15.730411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8071ec6-0ebb-475a-91ae-41661e0e1602 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:15.736770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 462/917] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Sep 26 05:31:15 2022] GET /identity/v3/endpoints?service_id=5df2988ab57e491db664382ef5331005&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:15.746558 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a161290e-730c-4377-a1d0-97e396445a00 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:15.806813 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:15.921464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a161290e-730c-4377-a1d0-97e396445a00 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:15.961959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc004f-9650-49d5-8c30-d7b0c89be90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:15.962599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc004f-9650-49d5-8c30-d7b0c89be90c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:15.962869 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc004f-9650-49d5-8c30-d7b0c89be90c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:15.963134 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc004f-9650-49d5-8c30-d7b0c89be90c None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:15.963819 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc004f-9650-49d5-8c30-d7b0c89be90c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:15.965329 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc004f-9650-49d5-8c30-d7b0c89be90c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:15.973179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 456/918] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 26 05:31:15 2022] GET /identity/v3/services => generated 1497 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:31:16.143125 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cba658-240e-4cc2-8d4f-cf8b9fd35eb4 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:16.143586 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cba658-240e-4cc2-8d4f-cf8b9fd35eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:16.143818 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cba658-240e-4cc2-8d4f-cf8b9fd35eb4 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:16.145302 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 463/919] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:16.157182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f0d64-5287-41eb-a739-709678580682 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:16.157182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f0d64-5287-41eb-a739-709678580682 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:16.157182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f0d64-5287-41eb-a739-709678580682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:16.188573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-743f0d64-5287-41eb-a739-709678580682 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:16.191471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-743f0d64-5287-41eb-a739-709678580682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:16.310463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 457/920] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:16.654265 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751745f2-bb43-4d0a-b87b-edc374f29f7f None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:16.654811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751745f2-bb43-4d0a-b87b-edc374f29f7f None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:16.655067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751745f2-bb43-4d0a-b87b-edc374f29f7f None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:16.657857 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 464/921] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:16.666233 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c4b3c-860d-49e4-b175-d756b6634cb3 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:16.666720 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c4b3c-860d-49e4-b175-d756b6634cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:16.667270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c4b3c-860d-49e4-b175-d756b6634cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:16.697505 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-1b3c4b3c-860d-49e4-b175-d756b6634cb3 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:16.699579 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1b3c4b3c-860d-49e4-b175-d756b6634cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:16.734513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:16.791068 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751745f2-bb43-4d0a-b87b-edc374f29f7f None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:16.828910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 458/922] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:31:16.930020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751745f2-bb43-4d0a-b87b-edc374f29f7f None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:16.961828 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:16.968175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f43e6-0b6a-44ca-ae4c-1af8ff486b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:16.968803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f43e6-0b6a-44ca-ae4c-1af8ff486b01 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:16.969124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f43e6-0b6a-44ca-ae4c-1af8ff486b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:16.969485 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f43e6-0b6a-44ca-ae4c-1af8ff486b01 None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:16.970125 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f43e6-0b6a-44ca-ae4c-1af8ff486b01 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:16.972114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f43e6-0b6a-44ca-ae4c-1af8ff486b01 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:16.974889 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: WARNING keystone.server.flask.common [None req-fa2f43e6-0b6a-44ca-ae4c-1af8ff486b01 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 26 05:31:16.996711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.sql.core [None req-fa2f43e6-0b6a-44ca-ae4c-1af8ff486b01 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 26 05:31:16.996711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 26 05:31:16.996711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [parameters: {'id': '3d0d4a1521094736aa2b23900b4471e4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 26 05:31:16.996711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81663) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 26 05:31:17.015081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-fa2f43e6-0b6a-44ca-ae4c-1af8ff486b01 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3d0d4a1521094736aa2b23900b4471e4.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3d0d4a1521094736aa2b23900b4471e4. Sep 26 05:31:17.015081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:17.015081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 05:31:17.015081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 05:31:17.015081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 05:31:17.015081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 05:31:17.015081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 05:31:17.015081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 05:31:17.015081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 05:31:17.015081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application conn.query(q) Sep 26 05:31:17.015081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 05:31:17.015081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 05:31:17.015081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application result.read() Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 26 05:31:17.016618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 26 05:31:17.019458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 05:31:17.019458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application next(self.gen) Sep 26 05:31:17.019458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 26 05:31:17.019458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application yield resource Sep 26 05:31:17.019458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 05:31:17.019458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application next(self.gen) Sep 26 05:31:17.019458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 26 05:31:17.019458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self.session.rollback() Sep 26 05:31:17.019458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 26 05:31:17.019458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self.force_reraise() Sep 26 05:31:17.019458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 26 05:31:17.019458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise self.value Sep 26 05:31:17.019458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 26 05:31:17.019458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 26 05:31:17.019458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 26 05:31:17.019458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application session.commit() Sep 26 05:31:17.019458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 26 05:31:17.024620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 26 05:31:17.024620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 26 05:31:17.024620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._prepare_impl() Sep 26 05:31:17.024620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 26 05:31:17.024620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self.session.flush() Sep 26 05:31:17.024620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 26 05:31:17.024620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._flush(objects) Sep 26 05:31:17.024620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 26 05:31:17.024620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 26 05:31:17.024620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 26 05:31:17.024620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application compat.raise_( Sep 26 05:31:17.024620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 05:31:17.024620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception Sep 26 05:31:17.024620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 26 05:31:17.024620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application flush_context.execute() Sep 26 05:31:17.024620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 26 05:31:17.024620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rec.execute(self) Sep 26 05:31:17.026675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 26 05:31:17.026675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 26 05:31:17.026675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 26 05:31:17.026675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 26 05:31:17.026675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 26 05:31:17.026675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 26 05:31:17.026675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 26 05:31:17.026675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 26 05:31:17.026675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 26 05:31:17.026675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 26 05:31:17.026675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 26 05:31:17.026675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 26 05:31:17.026675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 26 05:31:17.026675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 26 05:31:17.026675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 26 05:31:17.028358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 26 05:31:17.028358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 05:31:17.028358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception Sep 26 05:31:17.028358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 05:31:17.028358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 05:31:17.028358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 05:31:17.028358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 05:31:17.028358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 05:31:17.028358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 05:31:17.028358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 05:31:17.028358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application conn.query(q) Sep 26 05:31:17.028358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 05:31:17.028358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 05:31:17.028358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 05:31:17.028358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application result.read() Sep 26 05:31:17.028358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 05:31:17.028358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 05:31:17.031153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 05:31:17.031153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 05:31:17.031153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 05:31:17.031153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 05:31:17.031153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 05:31:17.031153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 05:31:17.031153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 26 05:31:17.031153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 26 05:31:17.031153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [parameters: {'id': '3d0d4a1521094736aa2b23900b4471e4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 26 05:31:17.031153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 26 05:31:17.031153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:17.031153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:17.031153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:17.031153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:17.031153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 26 05:31:17.034149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 26 05:31:17.034149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 26 05:31:17.034149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 26 05:31:17.034149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 26 05:31:17.034149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:17.034149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:17.034149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:17.034149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:17.034149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:17.034149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:17.034149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:17.034149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:17.034149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:17.034149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:17.034149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:17.034149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:17.034149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:17.037534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:17.037534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 26 05:31:17.037534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 26 05:31:17.037534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:17.037534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:17.037534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 26 05:31:17.037534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 26 05:31:17.037534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3d0d4a1521094736aa2b23900b4471e4. Sep 26 05:31:17.037534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:17.037534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 465/923] 10.222.0.57 () {60 vars in 1140 bytes} [Mon Sep 26 05:31:16 2022] POST /identity/v3/projects => generated 262 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:17.039490 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c4b3c-860d-49e4-b175-d756b6634cb3 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:17.202657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f43e6-0b6a-44ca-ae4c-1af8ff486b01 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:17.230753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c4b3c-860d-49e4-b175-d756b6634cb3 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:17.293180 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73931139-37ce-40e0-8b73-d61b76a4acbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:17.294235 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73931139-37ce-40e0-8b73-d61b76a4acbc None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:17.294714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73931139-37ce-40e0-8b73-d61b76a4acbc None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:17.295169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73931139-37ce-40e0-8b73-d61b76a4acbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:17.307688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73931139-37ce-40e0-8b73-d61b76a4acbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:17.310709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73931139-37ce-40e0-8b73-d61b76a4acbc None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:17.333746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-73931139-37ce-40e0-8b73-d61b76a4acbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:17.333746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:17.333746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:17.333746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:17.333746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:17.333746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:17.333746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:17.333746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:17.333746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:17.333746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:17.333746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:17.333746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:17.333746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:31:17.333746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:31:17.333746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:31:17.334761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:31:17.334761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:17.334761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:17.334761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:17.334761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:17.334761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:17.334761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:17.334761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:17.334761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:17.334761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:17.334761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:17.334761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:17.334761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:17.334761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:17.334761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:17.334761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:17.334761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:17.335689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:31:17.335689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:31:17.335689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:31:17.335689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:31:17.335689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:31:17.335689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:31:17.335689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:17.335689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:17.336407 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 459/924] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:31:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:17.346836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73931139-37ce-40e0-8b73-d61b76a4acbc None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:17.353492 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f43e6-0b6a-44ca-ae4c-1af8ff486b01 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:17.366354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:17.392821 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd4b53-49a0-4f93-8e41-4f59d7515d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:17.393885 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4b53-49a0-4f93-8e41-4f59d7515d7b None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:17.394365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4b53-49a0-4f93-8e41-4f59d7515d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:17.394831 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd4b53-49a0-4f93-8e41-4f59d7515d7b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:17.407956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd4b53-49a0-4f93-8e41-4f59d7515d7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:31:17.413806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-4bbd4b53-49a0-4f93-8e41-4f59d7515d7b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:31:17.413806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:17.413806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:17.413806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:17.413806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:17.413806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:17.413806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:17.413806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:17.413806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:17.413806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:17.413806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:17.413806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:17.413806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:31:17.413806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:31:17.413806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:31:17.413806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:31:17.414775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:31:17.414775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:31:17.414775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:31:17.414775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:31:17.414775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:17.414775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:17.414775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:17.414775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:17.414775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:17.414775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:17.414775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:17.414775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:17.414775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:17.414775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:17.414775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:17.414775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:17.415629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:17.415629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:17.415629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:17.415629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:17.415629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:31:17.415629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:31:17.415629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:31:17.415629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:31:17.415629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:31:17.415629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:31:17.415629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:31:17.415629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:17.418155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 466/925] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Sep 26 05:31:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:17.425357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd4b53-49a0-4f93-8e41-4f59d7515d7b None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:17.471961 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73931139-37ce-40e0-8b73-d61b76a4acbc None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:17.485438 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:17.514130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d72c1c-07cd-4230-bde8-7deed89635a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:17.515308 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d72c1c-07cd-4230-bde8-7deed89635a5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:17.515790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d72c1c-07cd-4230-bde8-7deed89635a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:17.516308 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d72c1c-07cd-4230-bde8-7deed89635a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:17.517248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d72c1c-07cd-4230-bde8-7deed89635a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:17.517765 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d72c1c-07cd-4230-bde8-7deed89635a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:17.519453 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d72c1c-07cd-4230-bde8-7deed89635a5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:17.550915 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 460/926] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 26 05:31:17 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:17.561200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd4b53-49a0-4f93-8e41-4f59d7515d7b None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:17.603274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bb9e8-358a-4d06-8f23-a4ad4085341c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:17.603912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bb9e8-358a-4d06-8f23-a4ad4085341c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:17.604191 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bb9e8-358a-4d06-8f23-a4ad4085341c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:17.604540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bb9e8-358a-4d06-8f23-a4ad4085341c None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:17.605273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bb9e8-358a-4d06-8f23-a4ad4085341c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:17.605648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bb9e8-358a-4d06-8f23-a4ad4085341c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:17.607287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bb9e8-358a-4d06-8f23-a4ad4085341c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:17.625337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 467/927] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Sep 26 05:31:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:17.627935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d72c1c-07cd-4230-bde8-7deed89635a5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:17.724664 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:17.761701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d72c1c-07cd-4230-bde8-7deed89635a5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:17.804501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23f803-1d6a-441a-a93f-ae851ba1b526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:17.805196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f803-1d6a-441a-a93f-ae851ba1b526 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:17.805477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f803-1d6a-441a-a93f-ae851ba1b526 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:17.805753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23f803-1d6a-441a-a93f-ae851ba1b526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:17.818856 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23f803-1d6a-441a-a93f-ae851ba1b526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:17.820360 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23f803-1d6a-441a-a93f-ae851ba1b526 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:17.840130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-fe23f803-1d6a-441a-a93f-ae851ba1b526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:17.840130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:17.840130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:17.840130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:17.840130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:17.840130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:17.840130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:17.840130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:17.840130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:17.840130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:17.840130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:17.840130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:17.840130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:17.840130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:17.840130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:17.840130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:17.841304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:17.841304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:17.841304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:17.841304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:17.841304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:17.841304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:17.841304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:17.841304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:17.841304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:17.841304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:17.841304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:17.841304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:17.841304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:17.841304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:17.841304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:17.841304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:17.841304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:17.842196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:17.842196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:17.842196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:17.845430 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 461/928] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:17.849895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bb9e8-358a-4d06-8f23-a4ad4085341c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:17.960484 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bb9e8-358a-4d06-8f23-a4ad4085341c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:17.971132 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:18.001030 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6edc9-3d8e-4b55-b9b1-55601c2501bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:18.002081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6edc9-3d8e-4b55-b9b1-55601c2501bb None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:18.002563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6edc9-3d8e-4b55-b9b1-55601c2501bb None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:18.003055 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6edc9-3d8e-4b55-b9b1-55601c2501bb None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:18.003928 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6edc9-3d8e-4b55-b9b1-55601c2501bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:18.004445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6edc9-3d8e-4b55-b9b1-55601c2501bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:18.006089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6edc9-3d8e-4b55-b9b1-55601c2501bb None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:18.020691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 468/929] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:18.026607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23f803-1d6a-441a-a93f-ae851ba1b526 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:18.153876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23f803-1d6a-441a-a93f-ae851ba1b526 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:18.185567 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:18.196373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f02a8-601f-4144-b2de-48cf15bb4468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:18.197041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f02a8-601f-4144-b2de-48cf15bb4468 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:18.197744 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f02a8-601f-4144-b2de-48cf15bb4468 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:18.198018 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f02a8-601f-4144-b2de-48cf15bb4468 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:18.207705 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f02a8-601f-4144-b2de-48cf15bb4468 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:18.209306 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f02a8-601f-4144-b2de-48cf15bb4468 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:18.217524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7203ba-4ca7-46c8-8b1d-d38484d3c819 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:18.217848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7203ba-4ca7-46c8-8b1d-d38484d3c819 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:18.218064 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7203ba-4ca7-46c8-8b1d-d38484d3c819 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:18.224473 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 469/930] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:18.224874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-579f02a8-601f-4144-b2de-48cf15bb4468 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:31:18.224874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:18.224874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:18.224874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:18.224874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:18.224874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:18.224874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:18.224874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:18.224874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:18.224874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:18.224874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:18.224874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:18.224874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:31:18.224874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:31:18.224874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:31:18.224874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:18.225854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:18.226865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:18.226865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:18.226865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:31:18.226865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:31:18.226865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:31:18.226865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:31:18.226865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:31:18.226865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:31:18.226865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:31:18.226865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:18.227407 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f88c84-069a-4512-9de4-824c0a9211cf None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:18.227482 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 462/931] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 26 05:31:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:18.229676 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f88c84-069a-4512-9de4-824c0a9211cf None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:18.229969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f88c84-069a-4512-9de4-824c0a9211cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:18.236625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f02a8-601f-4144-b2de-48cf15bb4468 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:18.256612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-32f88c84-069a-4512-9de4-824c0a9211cf None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:18.258368 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-32f88c84-069a-4512-9de4-824c0a9211cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:18.351127 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 470/932] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:18.409328 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f02a8-601f-4144-b2de-48cf15bb4468 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:18.471775 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b713546-a6b1-4673-ab52-0d0d5ee9092f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:18.472406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b713546-a6b1-4673-ab52-0d0d5ee9092f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:18.472684 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b713546-a6b1-4673-ab52-0d0d5ee9092f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:18.472959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b713546-a6b1-4673-ab52-0d0d5ee9092f None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:18.473752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b713546-a6b1-4673-ab52-0d0d5ee9092f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:18.474058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b713546-a6b1-4673-ab52-0d0d5ee9092f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:18.475553 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b713546-a6b1-4673-ab52-0d0d5ee9092f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:18.501321 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 463/933] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Sep 26 05:31:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:18.503857 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f88c84-069a-4512-9de4-824c0a9211cf None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:18.621945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b713546-a6b1-4673-ab52-0d0d5ee9092f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:18.632346 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:18.637197 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f88c84-069a-4512-9de4-824c0a9211cf None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:18.685935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:18.720129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00b36a-f89d-4a28-ae2e-dc673a764fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:18.722595 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00b36a-f89d-4a28-ae2e-dc673a764fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:18.723546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00b36a-f89d-4a28-ae2e-dc673a764fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:18.724419 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00b36a-f89d-4a28-ae2e-dc673a764fb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:18.747230 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00b36a-f89d-4a28-ae2e-dc673a764fb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:18.750334 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00b36a-f89d-4a28-ae2e-dc673a764fb4 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:18.780476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-4d00b36a-f89d-4a28-ae2e-dc673a764fb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:18.780476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:18.780476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:18.780476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:18.780476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:18.780476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:18.780476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:18.780476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:18.780476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:18.780476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:18.780476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:18.780476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:18.780476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:18.780476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:18.780476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:18.780476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:18.782386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:18.782386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:18.782386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:18.782386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:18.782386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:18.782386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:18.782386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:18.782386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:18.782386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:18.782386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:18.782386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:18.782386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:18.782386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:18.782386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:18.782386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:18.782386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:18.782386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:18.784850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:18.784850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:18.784850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:18.789306 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 471/934] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:18.793618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00b36a-f89d-4a28-ae2e-dc673a764fb4 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:18.812847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b713546-a6b1-4673-ab52-0d0d5ee9092f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:18.879466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096daad-6ffc-4ca6-a575-7dca650cb4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:18.880977 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096daad-6ffc-4ca6-a575-7dca650cb4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:18.881732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096daad-6ffc-4ca6-a575-7dca650cb4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:18.882429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096daad-6ffc-4ca6-a575-7dca650cb4e4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:18.883697 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096daad-6ffc-4ca6-a575-7dca650cb4e4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:18.886679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096daad-6ffc-4ca6-a575-7dca650cb4e4 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:18.903793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-8096daad-6ffc-4ca6-a575-7dca650cb4e4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:31:18.903793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:18.903793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:31:18.903793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:31:18.903793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:31:18.903793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:31:18.903793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:31:18.903793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:18.903793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:31:18.903793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:18.903793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:18.903793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:18.903793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:18.903793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:18.903793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:18.903793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:18.903793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:18.905735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:18.905735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:18.905735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:18.905735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:18.905735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:18.905735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:18.905735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:31:18.905735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:31:18.905735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:31:18.905735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:31:18.905735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:18.905735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:18.905735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:18.905735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:18.905735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:18.905735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:18.907401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:18.907401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:18.907401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:18.907401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:18.907401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:18.907401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:18.907401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:18.907401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:18.907401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:18.907401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:18.907401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:31:18.907401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:18.907401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:31:18.907401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:18.919149 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 464/935] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 26 05:31:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:18.924523 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096daad-6ffc-4ca6-a575-7dca650cb4e4 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:18.994462 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00b36a-f89d-4a28-ae2e-dc673a764fb4 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:19.059350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8939f2f-85f6-4144-b9c8-407b31bdbf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:19.060324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8939f2f-85f6-4144-b9c8-407b31bdbf68 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:19.060764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8939f2f-85f6-4144-b9c8-407b31bdbf68 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:19.061241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8939f2f-85f6-4144-b9c8-407b31bdbf68 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:19.062327 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8939f2f-85f6-4144-b9c8-407b31bdbf68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:19.062805 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8939f2f-85f6-4144-b9c8-407b31bdbf68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:19.065970 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8939f2f-85f6-4144-b9c8-407b31bdbf68 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:19.093270 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 472/936] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:19.098048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8939f2f-85f6-4144-b9c8-407b31bdbf68 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:19.118647 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096daad-6ffc-4ca6-a575-7dca650cb4e4 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:19.122931 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:19.141675 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:19.193624 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0388a22-fb9e-4104-aff2-9aae87742589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:19.194626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0388a22-fb9e-4104-aff2-9aae87742589 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:19.195102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0388a22-fb9e-4104-aff2-9aae87742589 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:19.195560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0388a22-fb9e-4104-aff2-9aae87742589 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:19.197038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0388a22-fb9e-4104-aff2-9aae87742589 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:19.197601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0388a22-fb9e-4104-aff2-9aae87742589 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:19.200930 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0388a22-fb9e-4104-aff2-9aae87742589 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:19.217229 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 465/937] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 26 05:31:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:19.224990 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0388a22-fb9e-4104-aff2-9aae87742589 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:19.321631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8939f2f-85f6-4144-b9c8-407b31bdbf68 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:19.388774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5877b2-8861-4ccc-8040-0993344624e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:19.389982 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5877b2-8861-4ccc-8040-0993344624e8 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:19.390546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5877b2-8861-4ccc-8040-0993344624e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:19.391118 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5877b2-8861-4ccc-8040-0993344624e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:19.404501 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5877b2-8861-4ccc-8040-0993344624e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:19.408207 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5877b2-8861-4ccc-8040-0993344624e8 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:19.416636 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0388a22-fb9e-4104-aff2-9aae87742589 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:19.426369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-ce5877b2-8861-4ccc-8040-0993344624e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:19.426369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:19.426369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:19.426369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:19.426369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:19.426369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:19.426369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:19.426369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:19.426369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:19.426369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:19.426369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:19.426369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:19.426369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:31:19.426369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:31:19.426369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:31:19.427657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:31:19.427657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:19.427657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:19.427657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:19.427657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:19.427657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:19.427657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:19.427657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:19.427657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:19.427657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:19.427657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:19.427657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:19.427657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:19.427657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:19.427657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:19.427657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:19.427657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:19.428873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:31:19.428873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:31:19.428873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:31:19.428873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:31:19.428873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:31:19.428873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:31:19.428873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:19.428873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:19.430765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 473/938] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:31:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:19.430765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db7a27-f744-448e-97ac-a5b0b0ea5dac None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:19.431313 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db7a27-f744-448e-97ac-a5b0b0ea5dac None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:19.431313 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db7a27-f744-448e-97ac-a5b0b0ea5dac None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:19.433079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 474/939] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:19.436177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db7a27-f744-448e-97ac-a5b0b0ea5dac None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:19.466080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb4134-663c-44e0-a537-7d2d394268aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:19.466850 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb4134-663c-44e0-a537-7d2d394268aa None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:19.467125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb4134-663c-44e0-a537-7d2d394268aa None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:19.467666 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb4134-663c-44e0-a537-7d2d394268aa None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:19.468195 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb4134-663c-44e0-a537-7d2d394268aa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:19.468593 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb4134-663c-44e0-a537-7d2d394268aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:19.469977 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb4134-663c-44e0-a537-7d2d394268aa None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:19.475841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 466/940] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 26 05:31:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:19.477110 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550513d7-6e84-45f9-93b0-ea65afb19400 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:19.477424 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550513d7-6e84-45f9-93b0-ea65afb19400 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:19.477931 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550513d7-6e84-45f9-93b0-ea65afb19400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:19.497715 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-550513d7-6e84-45f9-93b0-ea65afb19400 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:19.499161 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-550513d7-6e84-45f9-93b0-ea65afb19400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:19.537225 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:19.541801 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:19.594406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db7a27-f744-448e-97ac-a5b0b0ea5dac None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:19.605885 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 467/941] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:19.606776 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550513d7-6e84-45f9-93b0-ea65afb19400 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:19.661305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcdc50-d382-4dfb-b88c-8c4426e2ac10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:19.661967 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcdc50-d382-4dfb-b88c-8c4426e2ac10 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:19.662291 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcdc50-d382-4dfb-b88c-8c4426e2ac10 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:19.662625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcdc50-d382-4dfb-b88c-8c4426e2ac10 None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:19.663487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcdc50-d382-4dfb-b88c-8c4426e2ac10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:19.663854 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcdc50-d382-4dfb-b88c-8c4426e2ac10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:19.665547 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcdc50-d382-4dfb-b88c-8c4426e2ac10 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:19.693385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 475/942] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:31:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:19.700964 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcdc50-d382-4dfb-b88c-8c4426e2ac10 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:19.792906 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550513d7-6e84-45f9-93b0-ea65afb19400 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:19.843585 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114015f-576d-4b91-adb4-c4e346519ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:19.844801 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114015f-576d-4b91-adb4-c4e346519ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:19.845455 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114015f-576d-4b91-adb4-c4e346519ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:19.846151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114015f-576d-4b91-adb4-c4e346519ac2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:19.847553 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114015f-576d-4b91-adb4-c4e346519ac2 None admin] RBAC: Adding query filter params (interface=public, service_id=98aef52a9a9d4f00a0584f58ddd54ead, region_id=RegionOne) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:19.848120 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114015f-576d-4b91-adb4-c4e346519ac2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:19.850018 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3114015f-576d-4b91-adb4-c4e346519ac2 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:19.864823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 468/943] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Sep 26 05:31:19 2022] GET /identity/v3/endpoints?service_id=98aef52a9a9d4f00a0584f58ddd54ead&interface=public®ion_id=RegionOne => generated 194 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:19.867448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114015f-576d-4b91-adb4-c4e346519ac2 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:19.889028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcdc50-d382-4dfb-b88c-8c4426e2ac10 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:19.909962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:19.926103 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:19.935070 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52be8f-a4eb-4b99-b227-399c2d1bc75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:19.935568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52be8f-a4eb-4b99-b227-399c2d1bc75e None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:19.935848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52be8f-a4eb-4b99-b227-399c2d1bc75e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:19.936123 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52be8f-a4eb-4b99-b227-399c2d1bc75e None admin] PATH_INFO: `/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/0216bf67389e4627bf953b7dcc97a88c/roles/aedef758940042499c7698d83130db60` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:19.938744 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52be8f-a4eb-4b99-b227-399c2d1bc75e None admin] RBAC: Authorizing `identity:create_grant(project_id=5e44d601292b47b7b66d6f3efa80b450, user_id=0216bf67389e4627bf953b7dcc97a88c, role_id=aedef758940042499c7698d83130db60)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:19.940231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52be8f-a4eb-4b99-b227-399c2d1bc75e None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:19.960578 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 476/944] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:31:19 2022] PUT /identity/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/0216bf67389e4627bf953b7dcc97a88c/roles/aedef758940042499c7698d83130db60 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:31:19.961223 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52be8f-a4eb-4b99-b227-399c2d1bc75e None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:20.033095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3114015f-576d-4b91-adb4-c4e346519ac2 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:20.083409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1069776-8d6b-420c-b33c-308ef468e569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:20.084107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1069776-8d6b-420c-b33c-308ef468e569 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:20.084384 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1069776-8d6b-420c-b33c-308ef468e569 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:20.084708 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1069776-8d6b-420c-b33c-308ef468e569 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:20.085372 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1069776-8d6b-420c-b33c-308ef468e569 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:20.086819 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1069776-8d6b-420c-b33c-308ef468e569 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:20.093839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 469/945] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 26 05:31:19 2022] GET /identity/v3/services => generated 1497 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:31:20.099134 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a3eaa-2611-40da-99dd-c4cbb7a900f1 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:20.099134 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a3eaa-2611-40da-99dd-c4cbb7a900f1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:20.099134 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a3eaa-2611-40da-99dd-c4cbb7a900f1 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:20.102428 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 470/946] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:20.103874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d269a-636c-4682-a194-a8619f5a36c2 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:20.104155 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d269a-636c-4682-a194-a8619f5a36c2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:20.104485 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d269a-636c-4682-a194-a8619f5a36c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:20.138150 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a01d269a-636c-4682-a194-a8619f5a36c2 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:20.140648 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a01d269a-636c-4682-a194-a8619f5a36c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:20.144713 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52be8f-a4eb-4b99-b227-399c2d1bc75e None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:20.218574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78823e2-526f-481b-b73d-fde910809f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:20.219296 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78823e2-526f-481b-b73d-fde910809f11 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:20.219581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78823e2-526f-481b-b73d-fde910809f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:20.219849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78823e2-526f-481b-b73d-fde910809f11 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:20.220454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78823e2-526f-481b-b73d-fde910809f11 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:20.222784 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78823e2-526f-481b-b73d-fde910809f11 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:20.235286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-a78823e2-526f-481b-b73d-fde910809f11 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:31:20.235286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:20.235286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:31:20.235286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:31:20.235286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:31:20.235286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:31:20.235286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:31:20.235286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:20.235286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:31:20.235286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:20.235286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:20.235286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:20.235286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:20.235286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:20.235286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:20.235286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:20.237400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:20.237400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:20.237400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:20.237400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:20.237400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:20.237400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:20.237400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:20.237400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:31:20.237400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:31:20.237400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:31:20.237400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:31:20.237400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:20.237400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:20.237400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:20.237400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:20.237400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:20.237400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:20.240226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:20.240226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:20.240226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:20.240226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:20.240226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:20.240226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:20.240226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:20.240226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:20.240226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:20.240226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:20.240226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:31:20.240226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:20.240226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 05:31:20.240226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:20.240226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 477/947] 10.222.0.57 () {56 vars in 1132 bytes} [Mon Sep 26 05:31:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:20.244931 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78823e2-526f-481b-b73d-fde910809f11 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:20.295050 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 471/948] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:31:20.336607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:20.399905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78823e2-526f-481b-b73d-fde910809f11 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:20.444484 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27882445-9d7b-495d-9594-58fc99f3ffd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:20.445173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27882445-9d7b-495d-9594-58fc99f3ffd6 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:20.445453 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27882445-9d7b-495d-9594-58fc99f3ffd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:20.445729 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27882445-9d7b-495d-9594-58fc99f3ffd6 None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:20.446402 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27882445-9d7b-495d-9594-58fc99f3ffd6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:20.446704 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27882445-9d7b-495d-9594-58fc99f3ffd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:20.448193 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27882445-9d7b-495d-9594-58fc99f3ffd6 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:20.454706 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 478/949] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 26 05:31:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:20.460831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d269a-636c-4682-a194-a8619f5a36c2 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:20.498289 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:20.597497 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d269a-636c-4682-a194-a8619f5a36c2 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:20.659940 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b504d-b9e1-4c9f-bbac-d7f530d65859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:20.660955 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b504d-b9e1-4c9f-bbac-d7f530d65859 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:20.661467 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b504d-b9e1-4c9f-bbac-d7f530d65859 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:20.661919 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b504d-b9e1-4c9f-bbac-d7f530d65859 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:20.663125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b504d-b9e1-4c9f-bbac-d7f530d65859 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:20.663550 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b504d-b9e1-4c9f-bbac-d7f530d65859 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:20.666057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b504d-b9e1-4c9f-bbac-d7f530d65859 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:20.675514 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 472/950] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 26 05:31:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:20.681409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27882445-9d7b-495d-9594-58fc99f3ffd6 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:20.771942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b504d-b9e1-4c9f-bbac-d7f530d65859 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:20.815066 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27882445-9d7b-495d-9594-58fc99f3ffd6 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:20.857836 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593ae7d-d128-4b50-ba6a-d6611b0a7330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:20.858871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593ae7d-d128-4b50-ba6a-d6611b0a7330 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:20.859354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593ae7d-d128-4b50-ba6a-d6611b0a7330 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:20.860145 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593ae7d-d128-4b50-ba6a-d6611b0a7330 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:20.861091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593ae7d-d128-4b50-ba6a-d6611b0a7330 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:20.863244 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593ae7d-d128-4b50-ba6a-d6611b0a7330 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:20.868061 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:20.896448 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 479/951] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:31:20 2022] POST /identity/v3/endpoints => generated 347 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:31:20.975762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:21.001685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b504d-b9e1-4c9f-bbac-d7f530d65859 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:21.082551 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576000e0-4e73-495d-8817-9ffbe7c34f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:21.083657 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576000e0-4e73-495d-8817-9ffbe7c34f13 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:21.084180 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576000e0-4e73-495d-8817-9ffbe7c34f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:21.084641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576000e0-4e73-495d-8817-9ffbe7c34f13 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:21.085877 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576000e0-4e73-495d-8817-9ffbe7c34f13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:21.088618 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576000e0-4e73-495d-8817-9ffbe7c34f13 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:21.093454 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: WARNING keystone.server.flask.common [None req-576000e0-4e73-495d-8817-9ffbe7c34f13 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 26 05:31:21.115747 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.sql.core [None req-576000e0-4e73-495d-8817-9ffbe7c34f13 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 26 05:31:21.115747 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 26 05:31:21.115747 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [parameters: {'user_id': '47a42e3a4f144b10979c96fb2fcf9199', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 26 05:31:21.115747 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81664) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 26 05:31:21.128265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-576000e0-4e73-495d-8817-9ffbe7c34f13 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 26 05:31:21.128265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:21.128265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 05:31:21.128265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 05:31:21.128265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 05:31:21.128265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 05:31:21.128265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 05:31:21.128265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 05:31:21.128265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 05:31:21.128265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application conn.query(q) Sep 26 05:31:21.128265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 05:31:21.128265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 05:31:21.128265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 05:31:21.128265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application result.read() Sep 26 05:31:21.128265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 05:31:21.129253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application next(self.gen) Sep 26 05:31:21.130307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 26 05:31:21.130307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application yield resource Sep 26 05:31:21.130307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 05:31:21.130307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application next(self.gen) Sep 26 05:31:21.130307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 26 05:31:21.130307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self.session.rollback() Sep 26 05:31:21.130307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 26 05:31:21.130307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self.force_reraise() Sep 26 05:31:21.130307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 26 05:31:21.130307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise self.value Sep 26 05:31:21.130307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 26 05:31:21.130307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 26 05:31:21.130307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 26 05:31:21.130307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application session.commit() Sep 26 05:31:21.130307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 26 05:31:21.130307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 26 05:31:21.130307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 26 05:31:21.131232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._prepare_impl() Sep 26 05:31:21.131232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 26 05:31:21.131232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self.session.flush() Sep 26 05:31:21.131232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 26 05:31:21.131232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._flush(objects) Sep 26 05:31:21.131232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 26 05:31:21.131232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 26 05:31:21.131232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 26 05:31:21.131232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application compat.raise_( Sep 26 05:31:21.131232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 05:31:21.131232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception Sep 26 05:31:21.131232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 26 05:31:21.131232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application flush_context.execute() Sep 26 05:31:21.131232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 26 05:31:21.131232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rec.execute(self) Sep 26 05:31:21.131232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 26 05:31:21.131232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 26 05:31:21.132156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 26 05:31:21.132156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 26 05:31:21.132156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 26 05:31:21.132156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 26 05:31:21.132156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 26 05:31:21.132156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 26 05:31:21.132156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 26 05:31:21.132156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 26 05:31:21.132156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 26 05:31:21.132156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 26 05:31:21.132156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 26 05:31:21.132156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 26 05:31:21.132156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 26 05:31:21.132156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 26 05:31:21.132156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 05:31:21.132156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception Sep 26 05:31:21.133049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 05:31:21.133049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 05:31:21.133049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 05:31:21.133049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 05:31:21.133049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 05:31:21.133049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 05:31:21.133049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 05:31:21.133049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application conn.query(q) Sep 26 05:31:21.133049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 05:31:21.133049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 05:31:21.133049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 05:31:21.133049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application result.read() Sep 26 05:31:21.133049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 05:31:21.133049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 05:31:21.133049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 05:31:21.133049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 05:31:21.136055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 05:31:21.136055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 05:31:21.136055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 05:31:21.136055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 05:31:21.136055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 26 05:31:21.136055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 26 05:31:21.136055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [parameters: {'user_id': '47a42e3a4f144b10979c96fb2fcf9199', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 26 05:31:21.136055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 26 05:31:21.136055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:21.136055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:21.136055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:21.136055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:21.136055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:21.136055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:21.136055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:21.137979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:21.137979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:21.137979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:21.137979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:21.137979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:21.137979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:21.137979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:21.137979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 26 05:31:21.137979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 26 05:31:21.137979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:21.137979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:21.137979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:31:21.137979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:21.137979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:31:21.137979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:21.137979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 26 05:31:21.137979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 26 05:31:21.140471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 26 05:31:21.140471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 26 05:31:21.140471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 26 05:31:21.140471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 26 05:31:21.140471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 26 05:31:21.140471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:21.140471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 473/952] 10.222.0.57 () {60 vars in 1131 bytes} [Mon Sep 26 05:31:20 2022] POST /identity/v3/users => generated 160 bytes in 360 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:21.142980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593ae7d-d128-4b50-ba6a-d6611b0a7330 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:21.302816 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593ae7d-d128-4b50-ba6a-d6611b0a7330 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:21.363167 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dfac4-bcce-4ee2-9c03-8045ff22d9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:21.364634 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dfac4-bcce-4ee2-9c03-8045ff22d9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:21.365439 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dfac4-bcce-4ee2-9c03-8045ff22d9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:21.366157 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dfac4-bcce-4ee2-9c03-8045ff22d9e7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:21.381442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dfac4-bcce-4ee2-9c03-8045ff22d9e7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:21.384140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dfac4-bcce-4ee2-9c03-8045ff22d9e7 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:21.407536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-165dfac4-bcce-4ee2-9c03-8045ff22d9e7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 05:31:21.407536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:21.407536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:21.407536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:21.407536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:21.407536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:21.407536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:21.407536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:21.407536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:21.407536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:21.407536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:21.407536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:21.407536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:31:21.407536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:31:21.407536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:31:21.407536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:21.408837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:21.410738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:21.410738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:21.410738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:31:21.410738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:31:21.410738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:31:21.410738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:31:21.410738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:31:21.410738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:31:21.410738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 05:31:21.410738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:21.415175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 480/953] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:31:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:21.421564 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576000e0-4e73-495d-8817-9ffbe7c34f13 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:21.556573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576000e0-4e73-495d-8817-9ffbe7c34f13 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:21.626352 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d5b09-50fa-4d71-8d57-6c648e6f3754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:21.627898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d5b09-50fa-4d71-8d57-6c648e6f3754 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:21.628712 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d5b09-50fa-4d71-8d57-6c648e6f3754 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:21.629521 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d5b09-50fa-4d71-8d57-6c648e6f3754 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:21.631162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d5b09-50fa-4d71-8d57-6c648e6f3754 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:21.632113 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d5b09-50fa-4d71-8d57-6c648e6f3754 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:21.635751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d5b09-50fa-4d71-8d57-6c648e6f3754 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:21.716603 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 474/954] 10.222.0.57 () {56 vars in 1101 bytes} [Mon Sep 26 05:31:21 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:22.272542 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c47d3-5550-498c-a529-2b992a822542 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:22.273037 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c47d3-5550-498c-a529-2b992a822542 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:22.273563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c47d3-5550-498c-a529-2b992a822542 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:22.276227 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 481/955] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:22.283490 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb34187-8259-4398-95f1-1f61ab9b112d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:22.284087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb34187-8259-4398-95f1-1f61ab9b112d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:22.284575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb34187-8259-4398-95f1-1f61ab9b112d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:22.297180 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:22.317389 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-6fb34187-8259-4398-95f1-1f61ab9b112d None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:22.320098 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6fb34187-8259-4398-95f1-1f61ab9b112d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:22.448803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 475/956] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:31:22.715962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce3b13-f604-4e7a-8e9a-c522e225dc42 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:22.716861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce3b13-f604-4e7a-8e9a-c522e225dc42 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:22.716861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce3b13-f604-4e7a-8e9a-c522e225dc42 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:22.718025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 482/957] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:22.725153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0dea0-d68d-4576-bb24-6671cffd8b71 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:22.725677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0dea0-d68d-4576-bb24-6671cffd8b71 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:22.725910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0dea0-d68d-4576-bb24-6671cffd8b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:22.755022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-bfc0dea0-d68d-4576-bb24-6671cffd8b71 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:22.756259 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bfc0dea0-d68d-4576-bb24-6671cffd8b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:22.833166 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 476/958] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:31:22.923274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce3b13-f604-4e7a-8e9a-c522e225dc42 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:22.988712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:23.074247 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce3b13-f604-4e7a-8e9a-c522e225dc42 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:23.100714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0dea0-d68d-4576-bb24-6671cffd8b71 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:23.138309 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c501cd8-4448-40d9-8868-89e4ef095098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:23.139139 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c501cd8-4448-40d9-8868-89e4ef095098 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:23.139556 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c501cd8-4448-40d9-8868-89e4ef095098 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:23.140013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c501cd8-4448-40d9-8868-89e4ef095098 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:23.153460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c501cd8-4448-40d9-8868-89e4ef095098 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:31:23.159459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-7c501cd8-4448-40d9-8868-89e4ef095098 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:31:23.159459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:23.159459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:23.159459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:23.159459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:23.159459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:23.159459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:23.159459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:23.159459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:23.159459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:23.159459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:23.159459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:23.159459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:31:23.159459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:31:23.159459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:31:23.159459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:31:23.161371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:31:23.161371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:31:23.161371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:31:23.161371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:31:23.161371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:23.161371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:23.161371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:23.161371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:23.161371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:23.161371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:23.161371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:23.161371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:23.161371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:23.161371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:23.161371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:23.161371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:23.166133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:23.166133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:23.166133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:23.166133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:23.166133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:31:23.166133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:31:23.166133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:31:23.166133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:31:23.166133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:31:23.166133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:31:23.166133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:31:23.166133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:23.166133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 483/959] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Sep 26 05:31:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:23.167862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c501cd8-4448-40d9-8868-89e4ef095098 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:23.189485 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:23.252393 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0dea0-d68d-4576-bb24-6671cffd8b71 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:23.312990 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:23.319474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f1ea2-15fd-4937-bf66-8f58f3924473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:23.320862 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f1ea2-15fd-4937-bf66-8f58f3924473 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:23.321374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f1ea2-15fd-4937-bf66-8f58f3924473 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:23.321906 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f1ea2-15fd-4937-bf66-8f58f3924473 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:23.322613 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f1ea2-15fd-4937-bf66-8f58f3924473 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:23.324195 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f1ea2-15fd-4937-bf66-8f58f3924473 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:23.325045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c501cd8-4448-40d9-8868-89e4ef095098 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:23.335245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b30f1ea2-15fd-4937-bf66-8f58f3924473 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:31:23.335245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:23.335245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:31:23.335245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:31:23.335245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:31:23.335245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:31:23.335245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:31:23.335245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:23.335245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:31:23.335245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:23.335245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:23.335245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:23.335245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:23.335245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:23.335245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:23.335245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:23.335245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:23.341601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:23.341601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:23.341601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:23.341601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:23.341601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:23.341601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:23.341601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:31:23.341601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:31:23.341601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:31:23.341601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:31:23.341601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:23.341601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:23.341601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:23.341601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:23.341601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:23.341601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:23.343298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:23.343298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:23.343298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:23.343298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:23.343298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:23.343298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:23.343298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:23.343298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:23.343298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:23.343298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:23.343298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:31:23.343298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:23.343298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:31:23.343298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:23.343298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 477/960] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 26 05:31:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:23.347423 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f1ea2-15fd-4937-bf66-8f58f3924473 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:23.389884 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617efbbe-4722-47d1-88df-338bce99e5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:23.391001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617efbbe-4722-47d1-88df-338bce99e5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:23.391540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617efbbe-4722-47d1-88df-338bce99e5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:23.391993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617efbbe-4722-47d1-88df-338bce99e5e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:23.393267 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617efbbe-4722-47d1-88df-338bce99e5e0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:23.393998 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617efbbe-4722-47d1-88df-338bce99e5e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:23.396373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617efbbe-4722-47d1-88df-338bce99e5e0 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:23.415655 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 484/961] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Sep 26 05:31:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:23.417249 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617efbbe-4722-47d1-88df-338bce99e5e0 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:23.520374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f1ea2-15fd-4937-bf66-8f58f3924473 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:23.565459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:23.578636 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3007a4-5362-436f-bbaf-fd525fe5b8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:23.579601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3007a4-5362-436f-bbaf-fd525fe5b8bf None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:23.580033 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3007a4-5362-436f-bbaf-fd525fe5b8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:23.580457 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3007a4-5362-436f-bbaf-fd525fe5b8bf None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:23.581273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3007a4-5362-436f-bbaf-fd525fe5b8bf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:23.581760 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3007a4-5362-436f-bbaf-fd525fe5b8bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:23.584124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3007a4-5362-436f-bbaf-fd525fe5b8bf None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:23.592613 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 478/962] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 26 05:31:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:23.595078 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f4380-3e9c-4880-bb93-06b812914355 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:23.595565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f4380-3e9c-4880-bb93-06b812914355 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:23.596021 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f4380-3e9c-4880-bb93-06b812914355 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:23.599145 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 479/963] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:23.600194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617efbbe-4722-47d1-88df-338bce99e5e0 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:23.602889 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f4380-3e9c-4880-bb93-06b812914355 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:23.638079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7c2df-c042-4f89-911f-ada2fa15a6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:23.638724 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7c2df-c042-4f89-911f-ada2fa15a6b6 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:23.638999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7c2df-c042-4f89-911f-ada2fa15a6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:23.639363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7c2df-c042-4f89-911f-ada2fa15a6b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:23.651263 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7c2df-c042-4f89-911f-ada2fa15a6b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:23.652732 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7c2df-c042-4f89-911f-ada2fa15a6b6 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:23.668201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-c9f7c2df-c042-4f89-911f-ada2fa15a6b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:23.668201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:23.668201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:23.668201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:23.668201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:23.668201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:23.668201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:23.668201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:23.668201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:23.668201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:23.668201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:23.668201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:23.668201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:23.668201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:23.668201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:23.668201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:23.669324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:23.669324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:23.669324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:23.669324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:23.669324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:23.669324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:23.669324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:23.669324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:23.669324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:23.669324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:23.669324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:23.669324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:23.669324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:23.669324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:23.669324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:23.669324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:23.669324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:23.670266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:23.670266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:23.670266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:23.670266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 485/964] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:23.670683 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7d2fa-ddd8-4d07-a1ff-f685d6b20d69 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:23.671038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7d2fa-ddd8-4d07-a1ff-f685d6b20d69 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:23.671195 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7d2fa-ddd8-4d07-a1ff-f685d6b20d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:23.696017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-09c7d2fa-ddd8-4d07-a1ff-f685d6b20d69 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:23.697257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-09c7d2fa-ddd8-4d07-a1ff-f685d6b20d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:23.699564 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:23.735298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f4380-3e9c-4880-bb93-06b812914355 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:23.774599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db60a4a-d79e-484a-9d17-a5a195177b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:23.775143 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db60a4a-d79e-484a-9d17-a5a195177b06 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:23.775427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db60a4a-d79e-484a-9d17-a5a195177b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:23.775697 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db60a4a-d79e-484a-9d17-a5a195177b06 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:23.776390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db60a4a-d79e-484a-9d17-a5a195177b06 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:23.776697 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db60a4a-d79e-484a-9d17-a5a195177b06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:23.778162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db60a4a-d79e-484a-9d17-a5a195177b06 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:23.784560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 480/965] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 26 05:31:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:23.785470 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db60a4a-d79e-484a-9d17-a5a195177b06 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:23.806517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 486/966] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:23.806517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7d2fa-ddd8-4d07-a1ff-f685d6b20d69 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:23.869903 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:23.922453 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db60a4a-d79e-484a-9d17-a5a195177b06 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:23.956691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7d2fa-ddd8-4d07-a1ff-f685d6b20d69 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:23.969087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e23c27-b2d4-4ec4-8df1-567fc2f03094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:23.970224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23c27-b2d4-4ec4-8df1-567fc2f03094 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:23.970506 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23c27-b2d4-4ec4-8df1-567fc2f03094 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:23.970789 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23c27-b2d4-4ec4-8df1-567fc2f03094 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:23.972119 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e23c27-b2d4-4ec4-8df1-567fc2f03094 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:23.972436 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e23c27-b2d4-4ec4-8df1-567fc2f03094 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:23.974176 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e23c27-b2d4-4ec4-8df1-567fc2f03094 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:23.990347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 481/967] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:23.998819 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e23c27-b2d4-4ec4-8df1-567fc2f03094 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:23.999660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f265c6-a69a-4348-a413-a461f19822b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:24.000339 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f265c6-a69a-4348-a413-a461f19822b9 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:24.000602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f265c6-a69a-4348-a413-a461f19822b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:24.000943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f265c6-a69a-4348-a413-a461f19822b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:24.001908 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f265c6-a69a-4348-a413-a461f19822b9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:24.004616 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f265c6-a69a-4348-a413-a461f19822b9 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:24.018224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:24.037801 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 487/968] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:31:23 2022] POST /identity/v3/endpoints => generated 349 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:31:24.076494 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f265c6-a69a-4348-a413-a461f19822b9 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:24.156732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e23c27-b2d4-4ec4-8df1-567fc2f03094 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:24.180172 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:24.229330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809dc9a-74d2-4a69-a28d-17d1e8b13b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:24.230476 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809dc9a-74d2-4a69-a28d-17d1e8b13b07 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:24.230912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809dc9a-74d2-4a69-a28d-17d1e8b13b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:24.231401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809dc9a-74d2-4a69-a28d-17d1e8b13b07 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:24.241585 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809dc9a-74d2-4a69-a28d-17d1e8b13b07 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:24.243158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809dc9a-74d2-4a69-a28d-17d1e8b13b07 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:24.257981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f265c6-a69a-4348-a413-a461f19822b9 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:24.260674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-4809dc9a-74d2-4a69-a28d-17d1e8b13b07 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:31:24.260674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:24.260674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:24.260674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:24.260674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:24.260674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:24.260674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:24.260674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:24.260674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:24.260674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:24.260674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:24.260674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:24.260674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:31:24.260674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:31:24.260674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:31:24.260674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:24.261848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:24.262925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:24.262925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:24.262925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:31:24.262925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:31:24.262925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:31:24.262925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:31:24.262925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:31:24.262925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:31:24.262925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 05:31:24.262925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:24.262925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 482/969] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 26 05:31:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:24.270870 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809dc9a-74d2-4a69-a28d-17d1e8b13b07 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:24.315938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b18e91-df6b-43ad-9bae-53e5f5a6fdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:24.316867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b18e91-df6b-43ad-9bae-53e5f5a6fdb3 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:24.317301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b18e91-df6b-43ad-9bae-53e5f5a6fdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:24.317656 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b18e91-df6b-43ad-9bae-53e5f5a6fdb3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:24.318358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b18e91-df6b-43ad-9bae-53e5f5a6fdb3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:24.320639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b18e91-df6b-43ad-9bae-53e5f5a6fdb3 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:24.332568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-64b18e91-df6b-43ad-9bae-53e5f5a6fdb3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:31:24.332568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:24.332568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:31:24.332568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:31:24.332568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:31:24.332568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:31:24.332568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:31:24.332568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:24.332568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:31:24.332568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:24.332568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:24.332568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:24.332568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:24.332568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:24.332568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:24.332568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:24.332568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:24.334804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:24.334804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:24.334804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:24.334804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:24.334804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:24.334804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:24.334804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:31:24.334804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:31:24.334804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:31:24.334804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:31:24.334804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:24.334804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:24.334804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:24.334804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:24.334804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:24.334804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:24.336707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:24.336707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:24.336707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:24.336707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:24.336707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:24.336707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:24.336707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:24.336707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:24.336707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:24.336707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:24.336707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:31:24.336707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:24.336707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:31:24.336707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:24.336707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 488/970] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:31:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:24.344772 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b18e91-df6b-43ad-9bae-53e5f5a6fdb3 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:24.430397 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:24.461437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809dc9a-74d2-4a69-a28d-17d1e8b13b07 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:24.492512 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b18e91-df6b-43ad-9bae-53e5f5a6fdb3 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:24.515616 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fee5ed-e508-4aae-8459-b4544af17736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:24.516383 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fee5ed-e508-4aae-8459-b4544af17736 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:24.516739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fee5ed-e508-4aae-8459-b4544af17736 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:24.517072 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fee5ed-e508-4aae-8459-b4544af17736 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:24.518023 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fee5ed-e508-4aae-8459-b4544af17736 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:24.518406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fee5ed-e508-4aae-8459-b4544af17736 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:24.520008 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fee5ed-e508-4aae-8459-b4544af17736 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:24.545849 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 483/971] 10.222.0.57 () {56 vars in 1149 bytes} [Mon Sep 26 05:31:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:24.548396 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fee5ed-e508-4aae-8459-b4544af17736 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:24.562606 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09dfe43-487e-4a1e-9bdb-8e90e42e2786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:24.563585 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09dfe43-487e-4a1e-9bdb-8e90e42e2786 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:24.564022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09dfe43-487e-4a1e-9bdb-8e90e42e2786 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:24.564467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09dfe43-487e-4a1e-9bdb-8e90e42e2786 None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:24.565636 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09dfe43-487e-4a1e-9bdb-8e90e42e2786 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:24.566218 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09dfe43-487e-4a1e-9bdb-8e90e42e2786 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:24.568682 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09dfe43-487e-4a1e-9bdb-8e90e42e2786 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:24.576434 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 489/972] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 26 05:31:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:24.586319 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09dfe43-487e-4a1e-9bdb-8e90e42e2786 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:24.592535 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:24.689959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fee5ed-e508-4aae-8459-b4544af17736 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:24.729948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de291d-3fe5-4542-843a-4f3e32b0a9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:24.730592 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de291d-3fe5-4542-843a-4f3e32b0a9de None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:24.730879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de291d-3fe5-4542-843a-4f3e32b0a9de None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:24.731145 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de291d-3fe5-4542-843a-4f3e32b0a9de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:24.739425 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09dfe43-487e-4a1e-9bdb-8e90e42e2786 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:24.774420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:24.787840 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28081972-cf52-4490-9acf-5f486c038e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:24.788461 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28081972-cf52-4490-9acf-5f486c038e7d None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:24.788734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28081972-cf52-4490-9acf-5f486c038e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:24.788999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28081972-cf52-4490-9acf-5f486c038e7d None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:24.789718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28081972-cf52-4490-9acf-5f486c038e7d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:24.790022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28081972-cf52-4490-9acf-5f486c038e7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:24.791406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28081972-cf52-4490-9acf-5f486c038e7d None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:24.805371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 490/973] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 26 05:31:24 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:24.819721 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de291d-3fe5-4542-843a-4f3e32b0a9de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:24.821174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de291d-3fe5-4542-843a-4f3e32b0a9de None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:24.857408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:24.945692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 484/974] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:31:24 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 05:31:24.951825 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28081972-cf52-4490-9acf-5f486c038e7d None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:25.067532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28081972-cf52-4490-9acf-5f486c038e7d None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:25.108825 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57dc05-1507-4ee3-8a6d-613620780105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:25.109669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57dc05-1507-4ee3-8a6d-613620780105 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:25.109669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57dc05-1507-4ee3-8a6d-613620780105 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:25.109950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57dc05-1507-4ee3-8a6d-613620780105 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:25.122468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57dc05-1507-4ee3-8a6d-613620780105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:25.123922 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57dc05-1507-4ee3-8a6d-613620780105 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:25.144234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-5f57dc05-1507-4ee3-8a6d-613620780105 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:25.144234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:25.144234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:25.144234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:25.144234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:25.144234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:25.144234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:25.144234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:25.144234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:25.144234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:25.144234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:25.144234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:25.144234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:25.144234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:25.144234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:25.144234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:25.146785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:25.146785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:25.146785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:25.146785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:25.146785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:25.146785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:25.146785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:25.146785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:25.146785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:25.146785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:25.146785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:25.146785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:25.146785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:25.146785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:25.146785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:25.146785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:25.146785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:25.148816 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:25.148816 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:25.148816 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:25.148816 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 491/975] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:25.153851 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de291d-3fe5-4542-843a-4f3e32b0a9de None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:25.299559 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de291d-3fe5-4542-843a-4f3e32b0a9de None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:25.336979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:25.345300 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182102f-e9fb-4c0b-8618-697067d5f7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:25.346097 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182102f-e9fb-4c0b-8618-697067d5f7fe None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:25.346851 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182102f-e9fb-4c0b-8618-697067d5f7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:25.347252 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182102f-e9fb-4c0b-8618-697067d5f7fe None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:25.348043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182102f-e9fb-4c0b-8618-697067d5f7fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:25.348500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182102f-e9fb-4c0b-8618-697067d5f7fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:25.350101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182102f-e9fb-4c0b-8618-697067d5f7fe None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:25.368412 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 485/976] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:25.370092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57dc05-1507-4ee3-8a6d-613620780105 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:25.418676 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:25.492315 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57dc05-1507-4ee3-8a6d-613620780105 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:25.530438 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474c5b2-d490-42d9-a132-73f4dc8c6dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:25.531054 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474c5b2-d490-42d9-a132-73f4dc8c6dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:25.531330 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474c5b2-d490-42d9-a132-73f4dc8c6dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:25.531608 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474c5b2-d490-42d9-a132-73f4dc8c6dc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:25.538986 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474c5b2-d490-42d9-a132-73f4dc8c6dc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:25.540423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474c5b2-d490-42d9-a132-73f4dc8c6dc8 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:25.552363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-9474c5b2-d490-42d9-a132-73f4dc8c6dc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:25.552363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:25.552363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:25.552363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:25.552363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:25.552363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:25.552363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:25.552363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:25.552363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:25.552363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:25.552363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:25.552363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:25.552363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:31:25.552363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:31:25.552363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:31:25.554289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:31:25.554289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:25.554289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:25.554289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:25.554289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:25.554289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:25.554289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:25.554289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:25.554289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:25.554289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:25.554289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:25.554289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:25.554289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:25.554289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:25.554289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:25.554289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:25.554289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:25.556079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:31:25.556079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:31:25.556079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:31:25.556079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:31:25.556079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:31:25.556079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:31:25.556079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:25.556079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:25.556079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 492/977] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:31:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:25.567058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182102f-e9fb-4c0b-8618-697067d5f7fe None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:25.753578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182102f-e9fb-4c0b-8618-697067d5f7fe None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:25.813677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced3b0a-b286-4074-9085-5b54294a4144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:25.814884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced3b0a-b286-4074-9085-5b54294a4144 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:25.815368 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced3b0a-b286-4074-9085-5b54294a4144 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:25.815848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced3b0a-b286-4074-9085-5b54294a4144 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:25.817269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced3b0a-b286-4074-9085-5b54294a4144 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:25.817873 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced3b0a-b286-4074-9085-5b54294a4144 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:25.820574 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced3b0a-b286-4074-9085-5b54294a4144 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:25.852204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 486/978] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:31:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:25.861514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474c5b2-d490-42d9-a132-73f4dc8c6dc8 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:25.988219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:25.994712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474c5b2-d490-42d9-a132-73f4dc8c6dc8 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:26.037404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b99002-3bd1-405b-bdbb-0eb00e0a72df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:26.037908 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b99002-3bd1-405b-bdbb-0eb00e0a72df None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:26.038187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b99002-3bd1-405b-bdbb-0eb00e0a72df None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:26.038447 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b99002-3bd1-405b-bdbb-0eb00e0a72df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:26.039187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b99002-3bd1-405b-bdbb-0eb00e0a72df None admin] RBAC: Adding query filter params (role.id=aedef758940042499c7698d83130db60, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=0216bf67389e4627bf953b7dcc97a88c) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:26.039490 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b99002-3bd1-405b-bdbb-0eb00e0a72df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:26.040935 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b99002-3bd1-405b-bdbb-0eb00e0a72df None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:26.048230 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 493/979] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:31:25 2022] GET /identity/v3/role_assignments?user.id=0216bf67389e4627bf953b7dcc97a88c&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=aedef758940042499c7698d83130db60 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:27.164406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097f4bd-b3fd-45b4-b5b8-6554bcbd43c6 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:27.165114 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097f4bd-b3fd-45b4-b5b8-6554bcbd43c6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:27.165114 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097f4bd-b3fd-45b4-b5b8-6554bcbd43c6 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:27.166562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 487/980] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:27.172256 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a801ab-38bc-4292-8035-71d89af7f12b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:27.172789 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a801ab-38bc-4292-8035-71d89af7f12b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:27.172789 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a801ab-38bc-4292-8035-71d89af7f12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:27.192882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-12a801ab-38bc-4292-8035-71d89af7f12b None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:27.194111 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-12a801ab-38bc-4292-8035-71d89af7f12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:27.299072 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 494/981] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:27.511554 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97fda4-a6e6-4b00-9914-b1de2ce4a384 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:27.511832 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97fda4-a6e6-4b00-9914-b1de2ce4a384 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:27.511979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97fda4-a6e6-4b00-9914-b1de2ce4a384 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:27.513606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 488/982] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:27.519243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28602312-7434-4fc0-b85b-6a2a4278e6f3 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:27.519632 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28602312-7434-4fc0-b85b-6a2a4278e6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:27.519813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28602312-7434-4fc0-b85b-6a2a4278e6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:27.540684 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-28602312-7434-4fc0-b85b-6a2a4278e6f3 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:27.541964 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-28602312-7434-4fc0-b85b-6a2a4278e6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:27.589727 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97fda4-a6e6-4b00-9914-b1de2ce4a384 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:27.625958 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 495/983] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:27.691015 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:27.762612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97fda4-a6e6-4b00-9914-b1de2ce4a384 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:27.829355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcda2b-096f-4db2-a52c-ef7028bc5f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:27.830362 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcda2b-096f-4db2-a52c-ef7028bc5f5d None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:27.830839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcda2b-096f-4db2-a52c-ef7028bc5f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:27.831317 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcda2b-096f-4db2-a52c-ef7028bc5f5d None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:27.832606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffcda2b-096f-4db2-a52c-ef7028bc5f5d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:27.835518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffcda2b-096f-4db2-a52c-ef7028bc5f5d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:27.861240 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 489/984] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 26 05:31:27 2022] POST /identity/v3/services => generated 242 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:31:27.932045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28602312-7434-4fc0-b85b-6a2a4278e6f3 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:27.962777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:28.058679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28602312-7434-4fc0-b85b-6a2a4278e6f3 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:28.117328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31477154-f21f-42fd-a864-9eeb6c0ab753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:28.118002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31477154-f21f-42fd-a864-9eeb6c0ab753 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:28.118266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31477154-f21f-42fd-a864-9eeb6c0ab753 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:28.118530 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31477154-f21f-42fd-a864-9eeb6c0ab753 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:28.119130 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31477154-f21f-42fd-a864-9eeb6c0ab753 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:28.120593 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31477154-f21f-42fd-a864-9eeb6c0ab753 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:28.130815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-31477154-f21f-42fd-a864-9eeb6c0ab753 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:31:28.130815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:28.130815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:31:28.130815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:31:28.130815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:31:28.130815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:31:28.130815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:31:28.130815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:28.130815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:31:28.130815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:28.130815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:28.130815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:28.130815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:28.130815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:28.130815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:28.130815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:28.130815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:28.132597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:28.132597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:28.132597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:28.132597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:28.132597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:28.132597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:28.132597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:31:28.132597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:31:28.132597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:31:28.132597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:31:28.132597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:28.132597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:28.132597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:28.132597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:28.132597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:28.132597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:28.134336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:28.134336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:28.134336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:28.134336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:28.134336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:28.134336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:28.134336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:28.134336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:28.134336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:28.134336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:28.134336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:31:28.134336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:28.134336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:31:28.134336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:28.137468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 496/985] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 26 05:31:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:28.139099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcda2b-096f-4db2-a52c-ef7028bc5f5d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:28.309534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcda2b-096f-4db2-a52c-ef7028bc5f5d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:28.331652 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:28.351452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783f125-eeca-44c8-b786-b9eac85f6032 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:28.352062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783f125-eeca-44c8-b786-b9eac85f6032 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:28.352371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783f125-eeca-44c8-b786-b9eac85f6032 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:28.354939 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 497/986] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:28.364380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf07c53-d81e-45b5-bd74-36a99427893a None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:28.364712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf07c53-d81e-45b5-bd74-36a99427893a None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:28.365208 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf07c53-d81e-45b5-bd74-36a99427893a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:28.379719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba80c2-a078-4e7f-8a56-15f4b1d2c42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:28.380574 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba80c2-a078-4e7f-8a56-15f4b1d2c42e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:28.381039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba80c2-a078-4e7f-8a56-15f4b1d2c42e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:28.381542 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba80c2-a078-4e7f-8a56-15f4b1d2c42e None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:28.382348 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba80c2-a078-4e7f-8a56-15f4b1d2c42e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:28.382782 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba80c2-a078-4e7f-8a56-15f4b1d2c42e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:28.384357 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba80c2-a078-4e7f-8a56-15f4b1d2c42e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:28.390616 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 490/987] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 26 05:31:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:28.398000 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-fbf07c53-d81e-45b5-bd74-36a99427893a None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:28.399365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-fbf07c53-d81e-45b5-bd74-36a99427893a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:28.400428 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba80c2-a078-4e7f-8a56-15f4b1d2c42e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:28.533193 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 498/988] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:28 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:31:28.596005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba80c2-a078-4e7f-8a56-15f4b1d2c42e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:28.656676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0d030-5271-4dd3-be5d-1580207fe3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:28.657701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0d030-5271-4dd3-be5d-1580207fe3ef None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:28.658189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0d030-5271-4dd3-be5d-1580207fe3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:28.658647 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0d030-5271-4dd3-be5d-1580207fe3ef None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:28.659771 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0d030-5271-4dd3-be5d-1580207fe3ef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:28.660312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0d030-5271-4dd3-be5d-1580207fe3ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:28.662850 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0d030-5271-4dd3-be5d-1580207fe3ef None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:28.672299 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 491/989] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 26 05:31:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:28.681965 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf07c53-d81e-45b5-bd74-36a99427893a None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:28.706337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:28.806288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0d030-5271-4dd3-be5d-1580207fe3ef None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:28.858504 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf07c53-d81e-45b5-bd74-36a99427893a None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:28.874706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:28.909692 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da558e-63de-40cb-a7fb-9668a4cf3192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:28.910492 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da558e-63de-40cb-a7fb-9668a4cf3192 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:28.910920 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da558e-63de-40cb-a7fb-9668a4cf3192 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:28.911321 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da558e-63de-40cb-a7fb-9668a4cf3192 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:28.912209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da558e-63de-40cb-a7fb-9668a4cf3192 None admin] RBAC: Adding query filter params (interface=admin, service_id=98aef52a9a9d4f00a0584f58ddd54ead, region_id=RegionOne) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:28.912649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da558e-63de-40cb-a7fb-9668a4cf3192 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:28.914632 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da558e-63de-40cb-a7fb-9668a4cf3192 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:28.921602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 499/990] 10.222.0.57 () {56 vars in 1249 bytes} [Mon Sep 26 05:31:28 2022] GET /identity/v3/endpoints?service_id=98aef52a9a9d4f00a0584f58ddd54ead&interface=admin®ion_id=RegionOne => generated 193 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:28.928425 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da558e-63de-40cb-a7fb-9668a4cf3192 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:29.003665 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0d030-5271-4dd3-be5d-1580207fe3ef None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:29.022593 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:29.053286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da558e-63de-40cb-a7fb-9668a4cf3192 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:29.066147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf46c96-1d28-4a7d-85e3-a33fe9f84d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:29.067220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf46c96-1d28-4a7d-85e3-a33fe9f84d3e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:29.067702 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf46c96-1d28-4a7d-85e3-a33fe9f84d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:29.068176 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf46c96-1d28-4a7d-85e3-a33fe9f84d3e None admin] PATH_INFO: `/v3/services/image` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:29.069262 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf46c96-1d28-4a7d-85e3-a33fe9f84d3e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:29.071567 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf46c96-1d28-4a7d-85e3-a33fe9f84d3e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:29.087193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-dcf46c96-1d28-4a7d-85e3-a33fe9f84d3e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:31:29.087193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:29.087193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:31:29.087193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:31:29.087193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:31:29.087193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:31:29.087193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:31:29.087193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:29.087193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:31:29.087193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:29.087193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:29.087193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:29.087193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:29.087193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:29.087193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:29.087193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:29.087193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:29.088298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:29.088298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:29.088298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:29.088298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:29.088298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:29.088298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:29.088298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:31:29.088298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:31:29.088298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:31:29.088298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:31:29.088298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:29.088298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:29.088298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:29.088298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:29.088298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:29.088298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:29.089283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:29.089283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:29.089283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:29.089283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:29.089283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:29.089283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:29.089283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:29.089283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:29.089283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:29.089283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:29.089283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:31:29.089283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:29.089283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:31:29.089283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:29.090048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 492/991] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Sep 26 05:31:28 2022] GET /identity/v3/services/image => generated 86 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:29.097544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf46c96-1d28-4a7d-85e3-a33fe9f84d3e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:29.105181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02626f9-357a-4e70-8957-ef3ffdbcf48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:29.105819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02626f9-357a-4e70-8957-ef3ffdbcf48a None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:29.106095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02626f9-357a-4e70-8957-ef3ffdbcf48a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:29.106357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02626f9-357a-4e70-8957-ef3ffdbcf48a None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:29.106976 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02626f9-357a-4e70-8957-ef3ffdbcf48a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:29.108361 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02626f9-357a-4e70-8957-ef3ffdbcf48a None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:29.115496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 500/992] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 26 05:31:28 2022] GET /identity/v3/services => generated 1727 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:31:29.320819 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf46c96-1d28-4a7d-85e3-a33fe9f84d3e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:29.326735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:29.382974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08837c0-6349-49ca-b76a-98ec395c6a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:29.383926 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08837c0-6349-49ca-b76a-98ec395c6a46 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:29.384402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08837c0-6349-49ca-b76a-98ec395c6a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:29.384878 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08837c0-6349-49ca-b76a-98ec395c6a46 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:29.386193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08837c0-6349-49ca-b76a-98ec395c6a46 None admin] RBAC: Adding query filter params (name=image) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:29.386740 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08837c0-6349-49ca-b76a-98ec395c6a46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:29.389307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08837c0-6349-49ca-b76a-98ec395c6a46 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:29.397794 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 493/993] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:29.403999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02626f9-357a-4e70-8957-ef3ffdbcf48a None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:29.589596 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02626f9-357a-4e70-8957-ef3ffdbcf48a None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:29.658147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf8391-6fb1-40b7-aac5-a8edcfb3f554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:29.659094 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf8391-6fb1-40b7-aac5-a8edcfb3f554 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:29.659575 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf8391-6fb1-40b7-aac5-a8edcfb3f554 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:29.660060 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf8391-6fb1-40b7-aac5-a8edcfb3f554 None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:29.661246 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf8391-6fb1-40b7-aac5-a8edcfb3f554 None admin] RBAC: Adding query filter params (type=image) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:29.664078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf8391-6fb1-40b7-aac5-a8edcfb3f554 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:29.664458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf8391-6fb1-40b7-aac5-a8edcfb3f554 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:29.681258 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 501/994] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:30.269518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36450c-407b-455d-98f9-f64c859829c8 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:30.269835 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36450c-407b-455d-98f9-f64c859829c8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:30.270198 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36450c-407b-455d-98f9-f64c859829c8 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:30.271679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 494/995] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:30.277365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb72db9-0664-45a1-9c0a-89f253c73eab None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:30.277752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb72db9-0664-45a1-9c0a-89f253c73eab None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:30.277908 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb72db9-0664-45a1-9c0a-89f253c73eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:30.285782 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:30.300714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-9fb72db9-0664-45a1-9c0a-89f253c73eab None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:30.301972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9fb72db9-0664-45a1-9c0a-89f253c73eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:30.377693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 502/996] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:31:30.645649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36450c-407b-455d-98f9-f64c859829c8 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:30.794118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36450c-407b-455d-98f9-f64c859829c8 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:30.857630 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bc338-179c-46e5-9c65-5884924854b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:30.858739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bc338-179c-46e5-9c65-5884924854b5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:30.859241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bc338-179c-46e5-9c65-5884924854b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:30.859728 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bc338-179c-46e5-9c65-5884924854b5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:30.860820 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bc338-179c-46e5-9c65-5884924854b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:30.863459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bc338-179c-46e5-9c65-5884924854b5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:30.881419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-539bc338-179c-46e5-9c65-5884924854b5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:31:30.881419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:30.881419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:31:30.881419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:31:30.881419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:31:30.881419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:31:30.881419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:31:30.881419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:30.881419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:31:30.881419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:30.881419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:30.881419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:30.881419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:30.881419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:30.881419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:30.881419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:30.881419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:30.882509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:30.882509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:30.882509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:30.882509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:30.882509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:30.882509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:30.882509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:31:30.882509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:31:30.882509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:31:30.882509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:31:30.882509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:30.882509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:30.882509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:30.882509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:30.882509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:30.882509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:30.883381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:30.883381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:30.883381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:30.883381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:30.883381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:30.883381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:30.883381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:30.883381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:30.883381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:30.883381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:30.883381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:31:30.883381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:30.883381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:31:30.883381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:30.884163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 495/997] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:31:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:30.892083 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb72db9-0664-45a1-9c0a-89f253c73eab None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:31.005962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb72db9-0664-45a1-9c0a-89f253c73eab None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:31.045452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27589f-c7c0-40ba-afa9-4dc8ec3360ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:31.046238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27589f-c7c0-40ba-afa9-4dc8ec3360ea None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:31.046652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27589f-c7c0-40ba-afa9-4dc8ec3360ea None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:31.047084 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27589f-c7c0-40ba-afa9-4dc8ec3360ea None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:31.047860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e27589f-c7c0-40ba-afa9-4dc8ec3360ea None admin] RBAC: Adding query filter params (name=compute) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:31.048292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e27589f-c7c0-40ba-afa9-4dc8ec3360ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:31.050104 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e27589f-c7c0-40ba-afa9-4dc8ec3360ea None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:31.051558 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:31.061708 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 503/998] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 26 05:31:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:31.064273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bc338-179c-46e5-9c65-5884924854b5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:31.101804 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:31.189202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bc338-179c-46e5-9c65-5884924854b5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:31.238532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabf88a-e6bd-4104-97ff-58bcf6a84a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:31.238532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabf88a-e6bd-4104-97ff-58bcf6a84a91 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:31.238532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabf88a-e6bd-4104-97ff-58bcf6a84a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:31.238958 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabf88a-e6bd-4104-97ff-58bcf6a84a91 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:31.239379 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dabf88a-e6bd-4104-97ff-58bcf6a84a91 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:31.239687 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dabf88a-e6bd-4104-97ff-58bcf6a84a91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:31.241178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dabf88a-e6bd-4104-97ff-58bcf6a84a91 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:31.248689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 496/999] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 26 05:31:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:31.254666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27589f-c7c0-40ba-afa9-4dc8ec3360ea None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:31.377659 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27589f-c7c0-40ba-afa9-4dc8ec3360ea None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:31.416524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17517377-37bc-4800-829a-31ec17d07544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:31.417241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17517377-37bc-4800-829a-31ec17d07544 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:31.417506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17517377-37bc-4800-829a-31ec17d07544 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:31.417754 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17517377-37bc-4800-829a-31ec17d07544 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:31.418443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17517377-37bc-4800-829a-31ec17d07544 None admin] RBAC: Adding query filter params (interface=public, service_id=ebe1541111d34767aaf5c5de94477205, region_id=RegionOne) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:31.418746 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17517377-37bc-4800-829a-31ec17d07544 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:31.420112 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17517377-37bc-4800-829a-31ec17d07544 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:31.426482 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 504/1000] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Sep 26 05:31:31 2022] GET /identity/v3/endpoints?service_id=ebe1541111d34767aaf5c5de94477205&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:31.432994 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabf88a-e6bd-4104-97ff-58bcf6a84a91 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:31.466808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c06eb8-6959-491e-a6fa-eb4ce14c755e None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:31.467181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c06eb8-6959-491e-a6fa-eb4ce14c755e None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:31.467341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c06eb8-6959-491e-a6fa-eb4ce14c755e None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:31.468865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 505/1001] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:31.474633 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62027c0b-5d55-4d19-b161-7991eab9921d None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:31.474953 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62027c0b-5d55-4d19-b161-7991eab9921d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:31.475259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62027c0b-5d55-4d19-b161-7991eab9921d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:31.494223 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-62027c0b-5d55-4d19-b161-7991eab9921d None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:31.495728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-62027c0b-5d55-4d19-b161-7991eab9921d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:31.574732 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:31.584370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:31.589206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 506/1002] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:31:31.595590 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabf88a-e6bd-4104-97ff-58bcf6a84a91 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:31.638740 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69052a52-4223-4d29-8c2b-af6b6a92b4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:31.639254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69052a52-4223-4d29-8c2b-af6b6a92b4cb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:31.639530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69052a52-4223-4d29-8c2b-af6b6a92b4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:31.639821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69052a52-4223-4d29-8c2b-af6b6a92b4cb None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:31.640417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69052a52-4223-4d29-8c2b-af6b6a92b4cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:31.641856 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69052a52-4223-4d29-8c2b-af6b6a92b4cb None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:31.649365 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 497/1003] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 26 05:31:31 2022] GET /identity/v3/services => generated 1727 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:31:31.858587 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62027c0b-5d55-4d19-b161-7991eab9921d None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:31.986301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62027c0b-5d55-4d19-b161-7991eab9921d None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:32.032244 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c908457-7881-4612-a423-c0e0f4696c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:32.032925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c908457-7881-4612-a423-c0e0f4696c94 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:32.033236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c908457-7881-4612-a423-c0e0f4696c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:32.033509 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c908457-7881-4612-a423-c0e0f4696c94 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:32.034133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c908457-7881-4612-a423-c0e0f4696c94 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:32.035631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c908457-7881-4612-a423-c0e0f4696c94 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:32.046410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-2c908457-7881-4612-a423-c0e0f4696c94 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:31:32.046410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:32.046410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:31:32.046410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:31:32.046410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:31:32.046410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:31:32.046410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:31:32.046410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:32.046410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:31:32.046410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:32.046410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:32.046410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:32.046410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:32.046410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:32.046410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:32.046410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:32.046410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:32.048666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:32.048666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:32.048666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:32.048666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:32.048666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:32.048666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:32.048666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:31:32.048666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:31:32.048666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:31:32.048666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:31:32.048666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:32.048666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:32.048666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:32.048666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:32.048666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:32.048666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:32.050460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:32.050460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:32.050460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:32.050460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:32.050460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:32.050460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:32.050460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:32.050460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:32.050460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:32.050460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:32.050460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:31:32.050460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:32.050460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 05:31:32.050460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:32.053494 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 507/1004] 10.222.0.57 () {56 vars in 1126 bytes} [Mon Sep 26 05:31:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:32.054665 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69052a52-4223-4d29-8c2b-af6b6a92b4cb None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:32.197124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69052a52-4223-4d29-8c2b-af6b6a92b4cb None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:32.266018 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122653b7-c3e0-4959-af8b-bfe5f5606239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:32.266933 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122653b7-c3e0-4959-af8b-bfe5f5606239 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:32.267406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122653b7-c3e0-4959-af8b-bfe5f5606239 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:32.267879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122653b7-c3e0-4959-af8b-bfe5f5606239 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:32.268991 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122653b7-c3e0-4959-af8b-bfe5f5606239 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:32.269568 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122653b7-c3e0-4959-af8b-bfe5f5606239 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:32.272107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122653b7-c3e0-4959-af8b-bfe5f5606239 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:32.282336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 498/1005] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 26 05:31:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:32.291428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c908457-7881-4612-a423-c0e0f4696c94 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:32.396623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:32.479954 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c908457-7881-4612-a423-c0e0f4696c94 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:32.540978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8c3e5-0a64-41e7-ab94-196fd5d5b3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:32.541989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8c3e5-0a64-41e7-ab94-196fd5d5b3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:32.542438 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8c3e5-0a64-41e7-ab94-196fd5d5b3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:32.542908 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8c3e5-0a64-41e7-ab94-196fd5d5b3c6 None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:32.544087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8c3e5-0a64-41e7-ab94-196fd5d5b3c6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:32.544613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8c3e5-0a64-41e7-ab94-196fd5d5b3c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:32.547173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8c3e5-0a64-41e7-ab94-196fd5d5b3c6 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:32.564579 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 508/1006] 10.222.0.57 () {56 vars in 1122 bytes} [Mon Sep 26 05:31:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:32.566088 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122653b7-c3e0-4959-af8b-bfe5f5606239 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:32.609253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:32.747548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122653b7-c3e0-4959-af8b-bfe5f5606239 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:32.810852 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae7ca8-d7ca-4e08-a480-bcab4ac5f275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:32.811903 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae7ca8-d7ca-4e08-a480-bcab4ac5f275 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:32.812393 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae7ca8-d7ca-4e08-a480-bcab4ac5f275 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:32.812876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae7ca8-d7ca-4e08-a480-bcab4ac5f275 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:32.814156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae7ca8-d7ca-4e08-a480-bcab4ac5f275 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:32.816691 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae7ca8-d7ca-4e08-a480-bcab4ac5f275 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:32.853175 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 499/1007] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:31:32 2022] POST /identity/v3/endpoints => generated 325 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:31:33.070602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f5251-b087-4962-9ff5-0cbfb731f4a5 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:33.071128 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f5251-b087-4962-9ff5-0cbfb731f4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:33.071128 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f5251-b087-4962-9ff5-0cbfb731f4a5 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:33.072831 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 509/1008] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:33.078993 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9fe06-9511-42da-9668-514c1a58985c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:33.079303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9fe06-9511-42da-9668-514c1a58985c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:33.079559 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9fe06-9511-42da-9668-514c1a58985c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:33.100774 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-3ce9fe06-9511-42da-9668-514c1a58985c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:33.102081 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3ce9fe06-9511-42da-9668-514c1a58985c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:33.192679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 500/1009] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:33 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:33.465970 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f5251-b087-4962-9ff5-0cbfb731f4a5 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:33.613209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f5251-b087-4962-9ff5-0cbfb731f4a5 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:33.636610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:33.655272 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381f137-061e-47df-9b0a-f328ac9a8a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:33.656073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381f137-061e-47df-9b0a-f328ac9a8a04 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:33.656490 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381f137-061e-47df-9b0a-f328ac9a8a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:33.656888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381f137-061e-47df-9b0a-f328ac9a8a04 None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:33.657731 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381f137-061e-47df-9b0a-f328ac9a8a04 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:33.659265 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381f137-061e-47df-9b0a-f328ac9a8a04 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:33.681212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 510/1010] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 26 05:31:33 2022] POST /identity/v3/services => generated 242 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:31:34.154513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360c525-c6e7-4ba5-bac4-eb963b891d6d None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:34.155502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360c525-c6e7-4ba5-bac4-eb963b891d6d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:34.155502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360c525-c6e7-4ba5-bac4-eb963b891d6d None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:34.156745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 501/1011] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:34.163650 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc4b8e-9d7f-4459-869f-452665846723 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:34.164207 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc4b8e-9d7f-4459-869f-452665846723 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:34.164724 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc4b8e-9d7f-4459-869f-452665846723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:34.196422 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-99cc4b8e-9d7f-4459-869f-452665846723 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:34.198727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-99cc4b8e-9d7f-4459-869f-452665846723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:34.345195 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 511/1012] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:34 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:34.606435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360c525-c6e7-4ba5-bac4-eb963b891d6d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:34.620722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:34.737846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360c525-c6e7-4ba5-bac4-eb963b891d6d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:34.799775 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163e035-97a8-47f7-ba4e-27210e244ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:34.800837 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163e035-97a8-47f7-ba4e-27210e244ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:34.801347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163e035-97a8-47f7-ba4e-27210e244ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:34.801817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163e035-97a8-47f7-ba4e-27210e244ac5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:34.802877 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163e035-97a8-47f7-ba4e-27210e244ac5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:34.805396 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163e035-97a8-47f7-ba4e-27210e244ac5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:34.822437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-2163e035-97a8-47f7-ba4e-27210e244ac5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:31:34.822437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:34.822437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:31:34.822437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:31:34.822437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:31:34.822437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:31:34.822437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:31:34.822437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:34.822437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:31:34.822437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:34.822437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:34.822437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:34.822437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:34.822437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:34.822437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:34.822437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:34.822437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:34.824236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:34.824236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:34.824236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:34.824236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:34.824236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:34.824236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:34.824236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:31:34.824236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:31:34.824236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:31:34.824236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:31:34.824236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:34.824236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:34.824236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:34.824236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:34.824236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:34.824236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:34.826759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:34.826759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:34.826759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:34.826759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:34.826759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:34.826759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:34.826759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:34.826759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:34.826759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:34.826759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:34.826759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:31:34.826759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:34.826759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 05:31:34.826759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:34.829288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 502/1013] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:31:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:34.834901 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc4b8e-9d7f-4459-869f-452665846723 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:35.012426 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc4b8e-9d7f-4459-869f-452665846723 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:35.026834 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:35.078381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6f73f-30ab-4b68-944d-dd6533c8c352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:35.079505 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6f73f-30ab-4b68-944d-dd6533c8c352 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:35.080001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6f73f-30ab-4b68-944d-dd6533c8c352 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:35.080476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6f73f-30ab-4b68-944d-dd6533c8c352 None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:35.081628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6f73f-30ab-4b68-944d-dd6533c8c352 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:35.082270 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6f73f-30ab-4b68-944d-dd6533c8c352 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:35.084680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6f73f-30ab-4b68-944d-dd6533c8c352 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:35.100489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 512/1014] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 26 05:31:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:35.105351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163e035-97a8-47f7-ba4e-27210e244ac5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:35.252409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:35.315999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163e035-97a8-47f7-ba4e-27210e244ac5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:35.380989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4013857-5ec3-4d2e-9ba2-e9d153bc27c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:35.382044 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4013857-5ec3-4d2e-9ba2-e9d153bc27c3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:35.382516 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4013857-5ec3-4d2e-9ba2-e9d153bc27c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:35.382985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4013857-5ec3-4d2e-9ba2-e9d153bc27c3 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:35.384149 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4013857-5ec3-4d2e-9ba2-e9d153bc27c3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:35.384703 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4013857-5ec3-4d2e-9ba2-e9d153bc27c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:35.387567 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4013857-5ec3-4d2e-9ba2-e9d153bc27c3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:35.397534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 503/1015] 10.222.0.57 () {56 vars in 1112 bytes} [Mon Sep 26 05:31:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:35.406593 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6f73f-30ab-4b68-944d-dd6533c8c352 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:35.601113 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6f73f-30ab-4b68-944d-dd6533c8c352 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:35.670947 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47367157-d4ff-4b25-ae0d-0c2a27dc6727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:35.671990 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47367157-d4ff-4b25-ae0d-0c2a27dc6727 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:35.672468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47367157-d4ff-4b25-ae0d-0c2a27dc6727 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:35.672963 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47367157-d4ff-4b25-ae0d-0c2a27dc6727 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:35.674279 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47367157-d4ff-4b25-ae0d-0c2a27dc6727 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:35.676837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47367157-d4ff-4b25-ae0d-0c2a27dc6727 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:35.715260 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 513/1016] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:31:35 2022] POST /identity/v3/endpoints => generated 334 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:31:35.767738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d596c04-23d5-454d-8439-bb23c3b094e7 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:35.768123 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d596c04-23d5-454d-8439-bb23c3b094e7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:35.768295 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d596c04-23d5-454d-8439-bb23c3b094e7 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:35.769983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 504/1017] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:35.775891 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1943a-74ec-4738-8d62-4e66aca6b5d7 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:35.776381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1943a-74ec-4738-8d62-4e66aca6b5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:35.776606 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1943a-74ec-4738-8d62-4e66aca6b5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:35.796700 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-ffe1943a-74ec-4738-8d62-4e66aca6b5d7 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:35.797977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ffe1943a-74ec-4738-8d62-4e66aca6b5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:35.823453 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:35.905787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 514/1018] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:36.186386 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d596c04-23d5-454d-8439-bb23c3b094e7 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:36.201796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcbf0a-ef91-4492-98fd-4c07df263e0c None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:36.202095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcbf0a-ef91-4492-98fd-4c07df263e0c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:36.202438 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcbf0a-ef91-4492-98fd-4c07df263e0c None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:36.204131 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 515/1019] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:36.216726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6481e89-0995-42c4-a8ab-56edc35a5e9b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:36.217288 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6481e89-0995-42c4-a8ab-56edc35a5e9b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:36.217757 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6481e89-0995-42c4-a8ab-56edc35a5e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:36.247677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-c6481e89-0995-42c4-a8ab-56edc35a5e9b None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:36.248975 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c6481e89-0995-42c4-a8ab-56edc35a5e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:36.333803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d596c04-23d5-454d-8439-bb23c3b094e7 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:36.340923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 516/1020] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:31:36.395940 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725596f-bbbd-4469-b07b-51354beda3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:36.397002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725596f-bbbd-4469-b07b-51354beda3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:36.397522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725596f-bbbd-4469-b07b-51354beda3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:36.397999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725596f-bbbd-4469-b07b-51354beda3c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:36.400041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725596f-bbbd-4469-b07b-51354beda3c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:36.402770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725596f-bbbd-4469-b07b-51354beda3c9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:36.406036 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 505/1021] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:36.415618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6481e89-0995-42c4-a8ab-56edc35a5e9b None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:36.575647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:36.606325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725596f-bbbd-4469-b07b-51354beda3c9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:36.610560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:36.612810 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6481e89-0995-42c4-a8ab-56edc35a5e9b None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:36.679413 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd030fd-4e0f-4ab7-9e80-46f97762032c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:36.680432 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd030fd-4e0f-4ab7-9e80-46f97762032c None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:36.680932 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd030fd-4e0f-4ab7-9e80-46f97762032c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:36.681446 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd030fd-4e0f-4ab7-9e80-46f97762032c None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:36.682734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd030fd-4e0f-4ab7-9e80-46f97762032c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:36.685436 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd030fd-4e0f-4ab7-9e80-46f97762032c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:36.728186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725596f-bbbd-4469-b07b-51354beda3c9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:36.745167 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 517/1022] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 26 05:31:36 2022] POST /identity/v3/projects => generated 311 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:31:36.807303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a2b4b-b60d-4737-bd1c-096388443c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:36.808494 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a2b4b-b60d-4737-bd1c-096388443c77 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:36.808973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a2b4b-b60d-4737-bd1c-096388443c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:36.809487 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a2b4b-b60d-4737-bd1c-096388443c77 None admin] PATH_INFO: `/v3/services/image` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:36.810528 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a2b4b-b60d-4737-bd1c-096388443c77 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:36.812533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a2b4b-b60d-4737-bd1c-096388443c77 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:36.829821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-477a2b4b-b60d-4737-bd1c-096388443c77 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:31:36.829821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:36.829821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:31:36.829821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:31:36.829821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:31:36.829821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:31:36.829821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:31:36.829821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:36.829821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:31:36.829821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:36.829821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:36.829821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:36.829821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:36.829821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:36.829821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:36.829821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:36.829821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:36.831855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:36.831855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:36.831855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:36.831855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:36.831855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:36.831855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:36.831855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:31:36.831855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:31:36.831855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:31:36.831855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:31:36.831855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:36.831855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:36.831855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:36.831855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:36.831855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:36.831855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:36.835330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:36.835330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:36.835330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:36.835330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:36.835330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:36.835330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:36.835330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:36.835330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:36.835330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:36.835330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:36.835330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:31:36.835330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:36.835330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:31:36.835330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:36.835330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 506/1023] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Sep 26 05:31:36 2022] GET /identity/v3/services/image => generated 86 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:36.838917 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd030fd-4e0f-4ab7-9e80-46f97762032c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:36.963973 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd030fd-4e0f-4ab7-9e80-46f97762032c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:37.012713 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3fb26-a931-44cd-adb4-30d5a2a52fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:37.013582 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3fb26-a931-44cd-adb4-30d5a2a52fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:37.013862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3fb26-a931-44cd-adb4-30d5a2a52fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:37.014133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3fb26-a931-44cd-adb4-30d5a2a52fa0 None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:37.014785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3fb26-a931-44cd-adb4-30d5a2a52fa0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:37.015087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3fb26-a931-44cd-adb4-30d5a2a52fa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:37.016591 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3fb26-a931-44cd-adb4-30d5a2a52fa0 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:37.018092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:37.025673 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 518/1024] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:37.031016 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a2b4b-b60d-4737-bd1c-096388443c77 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:37.164437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:37.245212 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a2b4b-b60d-4737-bd1c-096388443c77 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:37.310279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e4f3f-e261-4044-ade8-37aee04860e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:37.311394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e4f3f-e261-4044-ade8-37aee04860e0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:37.311955 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e4f3f-e261-4044-ade8-37aee04860e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:37.312508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e4f3f-e261-4044-ade8-37aee04860e0 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:37.313753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e4f3f-e261-4044-ade8-37aee04860e0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:37.314467 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e4f3f-e261-4044-ade8-37aee04860e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:37.317335 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e4f3f-e261-4044-ade8-37aee04860e0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:37.332222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 507/1025] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:37.337879 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3fb26-a931-44cd-adb4-30d5a2a52fa0 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:37.533585 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3fb26-a931-44cd-adb4-30d5a2a52fa0 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:37.603379 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c506b0-1be3-475d-91bd-939fe96f8916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:37.604485 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c506b0-1be3-475d-91bd-939fe96f8916 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:37.604957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c506b0-1be3-475d-91bd-939fe96f8916 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:37.605566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c506b0-1be3-475d-91bd-939fe96f8916 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:37.606812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c506b0-1be3-475d-91bd-939fe96f8916 None admin] RBAC: Adding query filter params (interface=public, service_id=e8507ef57b774e399249603a95fa72ea, region_id=RegionOne) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:37.607355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c506b0-1be3-475d-91bd-939fe96f8916 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:37.610020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c506b0-1be3-475d-91bd-939fe96f8916 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:37.621016 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 519/1026] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Sep 26 05:31:37 2022] GET /identity/v3/endpoints?service_id=e8507ef57b774e399249603a95fa72ea&interface=public®ion_id=RegionOne => generated 194 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:37.630078 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e4f3f-e261-4044-ade8-37aee04860e0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:37.811473 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e4f3f-e261-4044-ade8-37aee04860e0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:37.864121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:37.874077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fce13-b13b-40a5-a5c8-10d2d1653c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:37.875424 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fce13-b13b-40a5-a5c8-10d2d1653c28 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:37.875870 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fce13-b13b-40a5-a5c8-10d2d1653c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:37.876303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fce13-b13b-40a5-a5c8-10d2d1653c28 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:37.877419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fce13-b13b-40a5-a5c8-10d2d1653c28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:37.879887 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fce13-b13b-40a5-a5c8-10d2d1653c28 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:37.893055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 508/1027] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 26 05:31:37 2022] GET /identity/v3/services => generated 1957 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:31:38.353019 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f2d1b-2d63-454c-9590-05ce4f2dbe94 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:38.353538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f2d1b-2d63-454c-9590-05ce4f2dbe94 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:38.354013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f2d1b-2d63-454c-9590-05ce4f2dbe94 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:38.356613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 520/1028] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:38.362346 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe4170-dc29-457d-ab71-726ad87edbd3 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:38.362691 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe4170-dc29-457d-ab71-726ad87edbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:38.362940 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe4170-dc29-457d-ab71-726ad87edbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:38.383061 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d6fe4170-dc29-457d-ab71-726ad87edbd3 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:38.384272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d6fe4170-dc29-457d-ab71-726ad87edbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:38.462107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 509/1029] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:31:38.728335 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f2d1b-2d63-454c-9590-05ce4f2dbe94 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:38.801636 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:38.850583 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f2d1b-2d63-454c-9590-05ce4f2dbe94 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:38.893188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18f5ef-ad34-4c1c-b400-90b46c4798e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:38.893188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18f5ef-ad34-4c1c-b400-90b46c4798e6 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:38.893188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18f5ef-ad34-4c1c-b400-90b46c4798e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:38.893188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18f5ef-ad34-4c1c-b400-90b46c4798e6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:38.907420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18f5ef-ad34-4c1c-b400-90b46c4798e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:31:38.912646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-6e18f5ef-ad34-4c1c-b400-90b46c4798e6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:31:38.912646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:38.912646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:38.912646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:38.912646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:38.912646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:38.912646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:38.912646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:38.912646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:38.912646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:38.912646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:38.912646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:38.912646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:31:38.912646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:31:38.912646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:31:38.912646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:31:38.914594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:31:38.914594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:31:38.914594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:31:38.914594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:31:38.914594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:38.914594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:38.914594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:38.914594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:38.914594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:38.914594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:38.914594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:38.914594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:38.914594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:38.914594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:38.914594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:38.914594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:38.916305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:38.916305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:38.916305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:38.916305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:38.916305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:31:38.916305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:31:38.916305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:31:38.916305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:31:38.916305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:31:38.916305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:31:38.916305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:31:38.916305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:38.924561 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 521/1030] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Sep 26 05:31:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:38.924775 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe4170-dc29-457d-ab71-726ad87edbd3 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:39.093375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ccbe4-e63c-48ba-ba8e-957125173c94 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:39.093653 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ccbe4-e63c-48ba-ba8e-957125173c94 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:39.093929 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ccbe4-e63c-48ba-ba8e-957125173c94 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:39.095508 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 522/1031] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:39.101598 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39c46c-37f0-4403-b464-ecfae05e3e94 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:39.101914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39c46c-37f0-4403-b464-ecfae05e3e94 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:39.102140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39c46c-37f0-4403-b464-ecfae05e3e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:39.108898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe4170-dc29-457d-ab71-726ad87edbd3 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:39.121438 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-8f39c46c-37f0-4403-b464-ecfae05e3e94 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:39.122665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-8f39c46c-37f0-4403-b464-ecfae05e3e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:39.149353 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:39.161958 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230b57d-5e9e-49d3-9552-60eb8e1c16ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:39.163247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230b57d-5e9e-49d3-9552-60eb8e1c16ea None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:39.163850 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230b57d-5e9e-49d3-9552-60eb8e1c16ea None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:39.164286 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230b57d-5e9e-49d3-9552-60eb8e1c16ea None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:39.165416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230b57d-5e9e-49d3-9552-60eb8e1c16ea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:39.165911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230b57d-5e9e-49d3-9552-60eb8e1c16ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:39.168379 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230b57d-5e9e-49d3-9552-60eb8e1c16ea None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:39.185202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 510/1032] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Sep 26 05:31:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:39.188625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230b57d-5e9e-49d3-9552-60eb8e1c16ea None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:39.231915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 523/1033] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:39.321269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230b57d-5e9e-49d3-9552-60eb8e1c16ea None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:39.365596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965234d-cd4f-4051-b89e-3a0203b8a1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:39.366231 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965234d-cd4f-4051-b89e-3a0203b8a1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:39.366509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965234d-cd4f-4051-b89e-3a0203b8a1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:39.366808 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965234d-cd4f-4051-b89e-3a0203b8a1b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:39.381344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965234d-cd4f-4051-b89e-3a0203b8a1b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:39.382818 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965234d-cd4f-4051-b89e-3a0203b8a1b3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:39.407742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-8965234d-cd4f-4051-b89e-3a0203b8a1b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:39.407742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:39.407742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:39.407742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:39.407742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:39.407742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:39.407742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:39.407742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:39.407742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:39.407742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:39.407742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:39.407742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:39.407742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:39.407742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:39.407742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:39.407742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:39.408847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:39.408847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:39.408847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:39.408847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:39.408847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:39.408847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:39.408847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:39.408847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:39.408847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:39.408847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:39.408847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:39.408847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:39.408847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:39.408847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:39.408847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:39.408847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:39.408847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:39.409956 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:39.409956 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:39.409956 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:39.413623 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 511/1034] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:39.418757 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39c46c-37f0-4403-b464-ecfae05e3e94 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:39.498090 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965234d-cd4f-4051-b89e-3a0203b8a1b3 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:39.515241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:39.554410 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:39.596524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39c46c-37f0-4403-b464-ecfae05e3e94 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:39.628357 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965234d-cd4f-4051-b89e-3a0203b8a1b3 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:39.653947 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b22924-6ab9-4f89-934a-5c19107a78e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:39.654985 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b22924-6ab9-4f89-934a-5c19107a78e7 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:39.655456 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b22924-6ab9-4f89-934a-5c19107a78e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:39.655924 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b22924-6ab9-4f89-934a-5c19107a78e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:39.657013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b22924-6ab9-4f89-934a-5c19107a78e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:39.657578 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b22924-6ab9-4f89-934a-5c19107a78e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:39.660254 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b22924-6ab9-4f89-934a-5c19107a78e7 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:39.673946 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6716d-50f7-445b-8225-00faa65202d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:39.674615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6716d-50f7-445b-8225-00faa65202d0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:39.674895 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6716d-50f7-445b-8225-00faa65202d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:39.675175 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6716d-50f7-445b-8225-00faa65202d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:39.677071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec6716d-50f7-445b-8225-00faa65202d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:39.678599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec6716d-50f7-445b-8225-00faa65202d0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:39.680543 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 512/1035] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:39.687447 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 524/1036] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:39.687694 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6716d-50f7-445b-8225-00faa65202d0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:39.688329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b22924-6ab9-4f89-934a-5c19107a78e7 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:39.843863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6716d-50f7-445b-8225-00faa65202d0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:39.853010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b22924-6ab9-4f89-934a-5c19107a78e7 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:39.857275 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:39.873105 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:39.890246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea7b72-478c-41e6-9712-6013753e2222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:39.890751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea7b72-478c-41e6-9712-6013753e2222 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:39.891463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea7b72-478c-41e6-9712-6013753e2222 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:39.891463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea7b72-478c-41e6-9712-6013753e2222 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:39.891964 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea7b72-478c-41e6-9712-6013753e2222 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:39.893468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea7b72-478c-41e6-9712-6013753e2222 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:39.909725 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d556532-3e13-405e-ab8f-d9a2d08d085c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:39.910423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d556532-3e13-405e-ab8f-d9a2d08d085c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:39.910705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d556532-3e13-405e-ab8f-d9a2d08d085c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:39.911032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d556532-3e13-405e-ab8f-d9a2d08d085c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:39.920322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d556532-3e13-405e-ab8f-d9a2d08d085c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:39.921821 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d556532-3e13-405e-ab8f-d9a2d08d085c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:39.924692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 513/1037] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:31:39 2022] POST /identity/v3/users => generated 285 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:31:39.943809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-7d556532-3e13-405e-ab8f-d9a2d08d085c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:31:39.943809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:39.943809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:39.943809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:39.943809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:39.943809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:39.943809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:39.943809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:39.943809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:39.943809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:39.943809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:39.943809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:39.943809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:31:39.943809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:31:39.943809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:31:39.943809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:39.945948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:39.955155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:39.955155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:39.955155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:31:39.955155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:31:39.955155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:31:39.955155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:31:39.955155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:31:39.955155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:31:39.955155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:31:39.955155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:39.955155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 525/1038] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 26 05:31:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:39.959858 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea7b72-478c-41e6-9712-6013753e2222 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:40.108381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea7b72-478c-41e6-9712-6013753e2222 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:40.157247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76770d-cfb2-4d44-9f09-0b127140e936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:40.158034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76770d-cfb2-4d44-9f09-0b127140e936 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:40.158440 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76770d-cfb2-4d44-9f09-0b127140e936 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:40.158831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76770d-cfb2-4d44-9f09-0b127140e936 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:40.159716 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76770d-cfb2-4d44-9f09-0b127140e936 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:40.160171 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76770d-cfb2-4d44-9f09-0b127140e936 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:40.162150 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76770d-cfb2-4d44-9f09-0b127140e936 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:40.190575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 514/1039] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 26 05:31:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:40.197926 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d556532-3e13-405e-ab8f-d9a2d08d085c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:40.325110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d556532-3e13-405e-ab8f-d9a2d08d085c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:40.365265 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bae5ea-532a-4407-a2eb-4737d3a0293f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:40.365896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bae5ea-532a-4407-a2eb-4737d3a0293f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:40.366194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bae5ea-532a-4407-a2eb-4737d3a0293f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:40.366458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bae5ea-532a-4407-a2eb-4737d3a0293f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:40.419819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:40.454789 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bae5ea-532a-4407-a2eb-4737d3a0293f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:40.456283 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bae5ea-532a-4407-a2eb-4737d3a0293f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:40.588759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 526/1040] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:31:40 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:31:40.598217 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76770d-cfb2-4d44-9f09-0b127140e936 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:40.643774 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:40.671597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68135691-6c7e-4c05-b6e6-1bbbc3e410e8 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:40.671839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68135691-6c7e-4c05-b6e6-1bbbc3e410e8 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:40.672181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68135691-6c7e-4c05-b6e6-1bbbc3e410e8 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:40.673719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 527/1041] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:40.679446 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211b42d-b856-472e-85d3-a83473ed40b5 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:40.679704 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211b42d-b856-472e-85d3-a83473ed40b5 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:40.680039 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211b42d-b856-472e-85d3-a83473ed40b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:40.699935 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-c211b42d-b856-472e-85d3-a83473ed40b5 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:40.701324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c211b42d-b856-472e-85d3-a83473ed40b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:40.772738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76770d-cfb2-4d44-9f09-0b127140e936 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:40.805656 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 528/1042] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:31:40.838077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172194c-5967-4c9a-a314-39d42fcb861e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:40.839063 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172194c-5967-4c9a-a314-39d42fcb861e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:40.839535 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172194c-5967-4c9a-a314-39d42fcb861e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:40.840002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172194c-5967-4c9a-a314-39d42fcb861e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:40.860354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172194c-5967-4c9a-a314-39d42fcb861e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:40.863123 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172194c-5967-4c9a-a314-39d42fcb861e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:40.888602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-0172194c-5967-4c9a-a314-39d42fcb861e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:40.888602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:40.888602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:40.888602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:40.888602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:40.888602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:40.888602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:40.888602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:40.888602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:40.888602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:40.888602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:40.888602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:40.888602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:40.888602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:40.888602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:40.888602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:40.889682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:40.889682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:40.889682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:40.889682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:40.889682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:40.889682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:40.889682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:40.889682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:40.889682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:40.889682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:40.889682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:40.889682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:40.889682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:40.889682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:40.889682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:40.889682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:40.889682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:40.890602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:40.890602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:40.890602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:40.893517 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 515/1043] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:40.898957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211b42d-b856-472e-85d3-a83473ed40b5 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:40.903178 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:41.028861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211b42d-b856-472e-85d3-a83473ed40b5 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:41.067688 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db4de9-bd68-43ee-b886-fd10de27bbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:41.068304 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db4de9-bd68-43ee-b886-fd10de27bbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:41.068580 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db4de9-bd68-43ee-b886-fd10de27bbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:41.070258 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db4de9-bd68-43ee-b886-fd10de27bbb3 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:41.070258 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db4de9-bd68-43ee-b886-fd10de27bbb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:41.070258 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db4de9-bd68-43ee-b886-fd10de27bbb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:41.070573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172194c-5967-4c9a-a314-39d42fcb861e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:41.071196 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db4de9-bd68-43ee-b886-fd10de27bbb3 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:41.097802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 529/1044] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:41.102126 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db4de9-bd68-43ee-b886-fd10de27bbb3 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:41.164981 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:41.195671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:41.196969 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172194c-5967-4c9a-a314-39d42fcb861e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:41.248823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc32d6-c3ef-4f9e-b19a-63f249f1fb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:41.249512 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc32d6-c3ef-4f9e-b19a-63f249f1fb4d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:41.249872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc32d6-c3ef-4f9e-b19a-63f249f1fb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:41.250626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc32d6-c3ef-4f9e-b19a-63f249f1fb4d None admin] PATH_INFO: `/v3/services/image` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:41.250787 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc32d6-c3ef-4f9e-b19a-63f249f1fb4d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:41.252419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc32d6-c3ef-4f9e-b19a-63f249f1fb4d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:41.267007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-3fdc32d6-c3ef-4f9e-b19a-63f249f1fb4d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:31:41.267007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:41.267007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:31:41.267007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:31:41.267007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:31:41.267007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:31:41.267007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:31:41.267007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:41.267007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:31:41.267007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:41.267007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:31:41.267007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:41.267007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:41.267007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:41.267007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:41.267007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:41.267007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:41.268698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:41.268698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:41.268698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:41.268698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:41.268698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:41.268698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:41.268698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:31:41.268698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:31:41.268698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:31:41.268698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:31:41.268698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:41.268698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:41.268698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:41.268698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:41.268698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:41.268698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:41.270374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:41.270374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:41.270374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:41.270374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:41.270374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:41.270374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:41.270374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:41.270374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:41.270374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:41.270374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:41.270374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:31:41.270374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:31:41.270374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 05:31:41.270374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:41.271423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db4de9-bd68-43ee-b886-fd10de27bbb3 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:41.276315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 516/1045] 10.222.0.57 () {56 vars in 1105 bytes} [Mon Sep 26 05:31:41 2022] GET /identity/v3/services/image => generated 86 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:41.278136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc32d6-c3ef-4f9e-b19a-63f249f1fb4d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:41.316171 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93da36-e601-444e-9d35-1108a01a1799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:41.317414 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93da36-e601-444e-9d35-1108a01a1799 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:41.317703 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93da36-e601-444e-9d35-1108a01a1799 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:41.317977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93da36-e601-444e-9d35-1108a01a1799 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:41.325566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93da36-e601-444e-9d35-1108a01a1799 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:41.327041 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93da36-e601-444e-9d35-1108a01a1799 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:41.339276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-ec93da36-e601-444e-9d35-1108a01a1799 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:41.339276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:41.339276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:41.339276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:41.339276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:41.339276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:41.339276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:41.339276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:41.339276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:41.339276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:41.339276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:41.339276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:41.339276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:31:41.339276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:31:41.339276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:31:41.340796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:31:41.340796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:41.340796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:41.340796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:41.340796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:41.340796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:41.340796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:41.340796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:41.340796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:41.340796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:41.340796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:41.340796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:41.340796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:41.340796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:41.340796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:41.340796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:41.340796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:41.342766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:31:41.342766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:31:41.342766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:31:41.342766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:31:41.342766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:31:41.342766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:31:41.342766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:41.342766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:41.342766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 530/1046] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:31:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:41.349650 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93da36-e601-444e-9d35-1108a01a1799 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:41.451041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc32d6-c3ef-4f9e-b19a-63f249f1fb4d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:41.499914 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331e00d-4436-45e6-9a2a-74fb8453ac97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:41.500539 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331e00d-4436-45e6-9a2a-74fb8453ac97 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:41.500651 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93da36-e601-444e-9d35-1108a01a1799 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:41.500814 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331e00d-4436-45e6-9a2a-74fb8453ac97 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:41.501080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331e00d-4436-45e6-9a2a-74fb8453ac97 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:41.501751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b331e00d-4436-45e6-9a2a-74fb8453ac97 None admin] RBAC: Adding query filter params (name=image) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:41.502057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b331e00d-4436-45e6-9a2a-74fb8453ac97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:41.503469 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b331e00d-4436-45e6-9a2a-74fb8453ac97 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:41.510599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 517/1047] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:41.516606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331e00d-4436-45e6-9a2a-74fb8453ac97 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:41.520483 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:41.549454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:41.574731 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a85301-2537-4d43-8867-cb6ef22cbd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:41.575895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a85301-2537-4d43-8867-cb6ef22cbd38 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:41.576614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a85301-2537-4d43-8867-cb6ef22cbd38 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:41.577028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a85301-2537-4d43-8867-cb6ef22cbd38 None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:41.578085 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a85301-2537-4d43-8867-cb6ef22cbd38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:41.578527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a85301-2537-4d43-8867-cb6ef22cbd38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:41.581736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a85301-2537-4d43-8867-cb6ef22cbd38 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:41.600284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 531/1048] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:31:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:41.607312 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a85301-2537-4d43-8867-cb6ef22cbd38 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:41.652084 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331e00d-4436-45e6-9a2a-74fb8453ac97 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:41.692933 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50e311-1a30-4740-8a57-d0c1305182bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:41.693686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50e311-1a30-4740-8a57-d0c1305182bd None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:41.694010 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50e311-1a30-4740-8a57-d0c1305182bd None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:41.694344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50e311-1a30-4740-8a57-d0c1305182bd None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:41.695047 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50e311-1a30-4740-8a57-d0c1305182bd None admin] RBAC: Adding query filter params (type=image) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:41.695415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50e311-1a30-4740-8a57-d0c1305182bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:41.696960 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50e311-1a30-4740-8a57-d0c1305182bd None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:41.705627 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 518/1049] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:41.709161 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50e311-1a30-4740-8a57-d0c1305182bd None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:41.735613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a85301-2537-4d43-8867-cb6ef22cbd38 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:41.773347 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dcd3d-07d6-41fd-bf1e-ab4b32f5853c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:41.773971 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dcd3d-07d6-41fd-bf1e-ab4b32f5853c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:41.774249 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dcd3d-07d6-41fd-bf1e-ab4b32f5853c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:41.774525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dcd3d-07d6-41fd-bf1e-ab4b32f5853c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:41.775266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6dcd3d-07d6-41fd-bf1e-ab4b32f5853c None admin] RBAC: Adding query filter params (role.id=aedef758940042499c7698d83130db60, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=92556048cbd04c409c3f4b6fd645627f) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:41.775566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6dcd3d-07d6-41fd-bf1e-ab4b32f5853c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:41.777102 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6dcd3d-07d6-41fd-bf1e-ab4b32f5853c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:41.785458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 532/1050] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:31:41 2022] GET /identity/v3/role_assignments?user.id=92556048cbd04c409c3f4b6fd645627f&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=aedef758940042499c7698d83130db60 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:41.826463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:41.877370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50e311-1a30-4740-8a57-d0c1305182bd None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:41.923126 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5693f5d-15b1-481b-90c5-30a5b7a5fc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:41.923816 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5693f5d-15b1-481b-90c5-30a5b7a5fc0f None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:41.924089 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5693f5d-15b1-481b-90c5-30a5b7a5fc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:41.924357 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5693f5d-15b1-481b-90c5-30a5b7a5fc0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:41.925063 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5693f5d-15b1-481b-90c5-30a5b7a5fc0f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:41.926561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5693f5d-15b1-481b-90c5-30a5b7a5fc0f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:41.953202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 519/1051] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:31:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:31:42.427133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb97ef2-6139-4f8e-a942-9481a86175cf None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:42.427699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb97ef2-6139-4f8e-a942-9481a86175cf None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:42.427699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb97ef2-6139-4f8e-a942-9481a86175cf None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:42.429289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 533/1052] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:42.435262 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfdfbb-4a60-4c27-bb08-3dfc89ae356d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:42.435798 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfdfbb-4a60-4c27-bb08-3dfc89ae356d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:42.435798 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfdfbb-4a60-4c27-bb08-3dfc89ae356d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:42.455629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-98bfdfbb-4a60-4c27-bb08-3dfc89ae356d None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:42.456813 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-98bfdfbb-4a60-4c27-bb08-3dfc89ae356d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:42.534266 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 520/1053] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:42.803252 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb97ef2-6139-4f8e-a942-9481a86175cf None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:42.861456 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:42.924154 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb97ef2-6139-4f8e-a942-9481a86175cf None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:42.962664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f9b2c-8ae8-40d6-bd4e-4806434e3f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:42.963473 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f9b2c-8ae8-40d6-bd4e-4806434e3f5c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:42.963906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f9b2c-8ae8-40d6-bd4e-4806434e3f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:42.964318 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f9b2c-8ae8-40d6-bd4e-4806434e3f5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:42.976813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f9b2c-8ae8-40d6-bd4e-4806434e3f5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:31:42.981995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-d76f9b2c-8ae8-40d6-bd4e-4806434e3f5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:31:42.981995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:42.981995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:42.981995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:42.981995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:42.981995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:42.981995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:42.981995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:42.981995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:42.981995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:42.981995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:42.981995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:42.981995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:31:42.981995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:31:42.981995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:31:42.981995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:31:42.983903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:31:42.983903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:31:42.983903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:31:42.983903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:31:42.983903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:42.983903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:42.983903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:42.983903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:42.983903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:42.983903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:42.983903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:42.983903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:42.983903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:42.983903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:42.983903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:42.983903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:42.986242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:42.986242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:42.986242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:42.986242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:42.986242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:31:42.986242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:31:42.986242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:31:42.986242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:31:42.986242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:31:42.986242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:31:42.986242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:31:42.986242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:42.989633 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 534/1054] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:31:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:42.992542 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfdfbb-4a60-4c27-bb08-3dfc89ae356d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:43.056713 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:43.128670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfdfbb-4a60-4c27-bb08-3dfc89ae356d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:43.174796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df90660-32b4-4032-bcbc-a30b111f712e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:43.175475 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df90660-32b4-4032-bcbc-a30b111f712e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:43.175791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df90660-32b4-4032-bcbc-a30b111f712e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:43.176132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df90660-32b4-4032-bcbc-a30b111f712e None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:43.176826 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df90660-32b4-4032-bcbc-a30b111f712e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:43.177125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df90660-32b4-4032-bcbc-a30b111f712e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:43.178633 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df90660-32b4-4032-bcbc-a30b111f712e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:43.189603 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 521/1055] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:31:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:43.196366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f9b2c-8ae8-40d6-bd4e-4806434e3f5c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:43.375615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f9b2c-8ae8-40d6-bd4e-4806434e3f5c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:43.388809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:43.438870 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d14aa2-49cd-4dc8-b93c-709c520b3f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:43.439935 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d14aa2-49cd-4dc8-b93c-709c520b3f8e None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:43.440410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d14aa2-49cd-4dc8-b93c-709c520b3f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:43.440863 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d14aa2-49cd-4dc8-b93c-709c520b3f8e None admin] PATH_INFO: `/v3/users/291cf39fa0e4457c956911cc8bb06c08` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:43.465686 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d14aa2-49cd-4dc8-b93c-709c520b3f8e None admin] RBAC: Authorizing `identity:get_user(user_id=291cf39fa0e4457c956911cc8bb06c08)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:43.468303 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d14aa2-49cd-4dc8-b93c-709c520b3f8e None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:43.476741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 535/1056] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:31:43 2022] GET /identity/v3/users/291cf39fa0e4457c956911cc8bb06c08 => generated 285 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:43.478113 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df90660-32b4-4032-bcbc-a30b111f712e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:43.589039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df90660-32b4-4032-bcbc-a30b111f712e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:43.601277 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:43.629597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66415394-f0b5-4938-907a-2aea654a5c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:43.630123 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66415394-f0b5-4938-907a-2aea654a5c20 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:43.630398 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66415394-f0b5-4938-907a-2aea654a5c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:43.630675 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66415394-f0b5-4938-907a-2aea654a5c20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:43.701463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66415394-f0b5-4938-907a-2aea654a5c20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:43.702921 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66415394-f0b5-4938-907a-2aea654a5c20 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:43.828310 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 522/1057] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:31:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:31:43.834252 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d14aa2-49cd-4dc8-b93c-709c520b3f8e None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:43.947006 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d14aa2-49cd-4dc8-b93c-709c520b3f8e None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:43.990487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410fba4-a65b-4041-8de4-70c36d4b8a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:43.991150 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410fba4-a65b-4041-8de4-70c36d4b8a08 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:43.991425 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410fba4-a65b-4041-8de4-70c36d4b8a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:43.991691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410fba4-a65b-4041-8de4-70c36d4b8a08 None admin] PATH_INFO: `/v3/projects/0cd9ee5a4a9247ddb49223aa91faad03` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:43.992873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410fba4-a65b-4041-8de4-70c36d4b8a08 None admin] RBAC: Authorizing `identity:get_project(project_id=0cd9ee5a4a9247ddb49223aa91faad03)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:43.994372 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410fba4-a65b-4041-8de4-70c36d4b8a08 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:43.996386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 536/1058] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:31:43 2022] GET /identity/v3/projects/0cd9ee5a4a9247ddb49223aa91faad03 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:44.002881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66415394-f0b5-4938-907a-2aea654a5c20 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:44.017470 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:44.082270 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dba6c6-9609-4d34-b964-bd078b451133 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:44.082642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dba6c6-9609-4d34-b964-bd078b451133 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:44.082980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dba6c6-9609-4d34-b964-bd078b451133 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:44.085915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 537/1059] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:44.091324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9b687-ea29-4368-a574-02af6ff98a72 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:44.091601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9b687-ea29-4368-a574-02af6ff98a72 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:44.091873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9b687-ea29-4368-a574-02af6ff98a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:44.112207 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-69a9b687-ea29-4368-a574-02af6ff98a72 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:44.113433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-69a9b687-ea29-4368-a574-02af6ff98a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:44.123975 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66415394-f0b5-4938-907a-2aea654a5c20 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:44.146743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:44.164405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd7a46-5269-4973-a797-1d251ea6c061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:44.165415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd7a46-5269-4973-a797-1d251ea6c061 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:44.165705 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd7a46-5269-4973-a797-1d251ea6c061 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:44.165994 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd7a46-5269-4973-a797-1d251ea6c061 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:44.166788 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd7a46-5269-4973-a797-1d251ea6c061 None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=0cd9ee5a4a9247ddb49223aa91faad03, user.id=291cf39fa0e4457c956911cc8bb06c08) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:44.167096 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd7a46-5269-4973-a797-1d251ea6c061 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:44.168589 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd7a46-5269-4973-a797-1d251ea6c061 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:44.185173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 523/1060] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:31:44 2022] GET /identity/v3/role_assignments?user.id=291cf39fa0e4457c956911cc8bb06c08&scope.project.id=0cd9ee5a4a9247ddb49223aa91faad03&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:44.212741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 538/1061] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:44.479637 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd7a46-5269-4973-a797-1d251ea6c061 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:44.567160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:44.599360 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd7a46-5269-4973-a797-1d251ea6c061 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:44.650008 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b395066-dd4a-4191-b210-82b91d1dbf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:44.650673 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b395066-dd4a-4191-b210-82b91d1dbf8b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:44.650951 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b395066-dd4a-4191-b210-82b91d1dbf8b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:44.651267 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b395066-dd4a-4191-b210-82b91d1dbf8b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:44.663014 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b395066-dd4a-4191-b210-82b91d1dbf8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:31:44.668562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-2b395066-dd4a-4191-b210-82b91d1dbf8b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:31:44.668562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:44.668562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:44.668562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:44.668562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:44.668562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:44.668562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:44.668562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:44.668562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:44.668562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:44.668562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:44.668562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:44.668562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:31:44.668562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:31:44.668562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:31:44.668562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:31:44.669615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:31:44.669615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:31:44.669615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:31:44.669615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:31:44.669615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:44.669615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:44.669615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:44.669615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:44.669615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:44.669615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:44.669615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:44.669615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:44.669615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:44.669615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:44.669615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:44.669615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:44.670488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:44.670488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:44.670488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:44.670488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:44.670488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:31:44.670488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:31:44.670488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:31:44.670488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:31:44.670488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:31:44.670488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:31:44.670488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:31:44.670488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:44.671907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 524/1062] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Sep 26 05:31:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:44.676017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9b687-ea29-4368-a574-02af6ff98a72 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:44.740587 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8fd42-d69b-42c1-a58b-9e86d6704426 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:44.740793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8fd42-d69b-42c1-a58b-9e86d6704426 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:44.741164 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8fd42-d69b-42c1-a58b-9e86d6704426 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:44.742464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 525/1063] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:44.748414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1058bf-7d94-495f-ac5b-09f11135db53 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:44.748712 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1058bf-7d94-495f-ac5b-09f11135db53 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:44.749024 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1058bf-7d94-495f-ac5b-09f11135db53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:44.768709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-1a1058bf-7d94-495f-ac5b-09f11135db53 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:44.769972 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1a1058bf-7d94-495f-ac5b-09f11135db53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:44.824379 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9b687-ea29-4368-a574-02af6ff98a72 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:44.862991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd635a1d-d645-4b06-bd1b-5222ad31a64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:44.863618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd635a1d-d645-4b06-bd1b-5222ad31a64d None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:44.863946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd635a1d-d645-4b06-bd1b-5222ad31a64d None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:44.864223 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd635a1d-d645-4b06-bd1b-5222ad31a64d None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:44.864859 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd635a1d-d645-4b06-bd1b-5222ad31a64d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:44.865242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd635a1d-d645-4b06-bd1b-5222ad31a64d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:44.866992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd635a1d-d645-4b06-bd1b-5222ad31a64d None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:44.880937 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 526/1064] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:31:44.881189 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 539/1065] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Sep 26 05:31:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:44.883015 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1058bf-7d94-495f-ac5b-09f11135db53 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:44.952174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:44.997438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1058bf-7d94-495f-ac5b-09f11135db53 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:45.035852 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96126e8-0091-4410-9e89-d8f768a25225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:45.036499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96126e8-0091-4410-9e89-d8f768a25225 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:45.036772 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96126e8-0091-4410-9e89-d8f768a25225 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:45.037037 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96126e8-0091-4410-9e89-d8f768a25225 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:45.049403 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96126e8-0091-4410-9e89-d8f768a25225 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:45.050945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96126e8-0091-4410-9e89-d8f768a25225 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:45.067354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b96126e8-0091-4410-9e89-d8f768a25225 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:45.067354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:45.067354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:45.067354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:45.067354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:45.067354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:45.067354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:45.067354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:45.067354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:45.067354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:45.067354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:45.067354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:45.067354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:45.067354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:45.067354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:45.067354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:45.068438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:45.068438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:45.068438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:45.068438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:45.068438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:45.068438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:45.068438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:45.068438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:45.068438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:45.068438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:45.068438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:45.068438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:45.068438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:45.068438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:45.068438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:45.068438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:45.068438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:45.070349 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:45.070349 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:45.070349 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:45.073748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 527/1066] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:45.074555 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd635a1d-d645-4b06-bd1b-5222ad31a64d None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:45.103658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:45.200241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd635a1d-d645-4b06-bd1b-5222ad31a64d None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:45.266029 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22260cd-0bdf-448e-b3d2-024a12c38c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:45.267665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22260cd-0bdf-448e-b3d2-024a12c38c62 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:45.268261 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22260cd-0bdf-448e-b3d2-024a12c38c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:45.268391 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96126e8-0091-4410-9e89-d8f768a25225 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:45.269169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22260cd-0bdf-448e-b3d2-024a12c38c62 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:45.270188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22260cd-0bdf-448e-b3d2-024a12c38c62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:45.270830 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22260cd-0bdf-448e-b3d2-024a12c38c62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:45.274021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22260cd-0bdf-448e-b3d2-024a12c38c62 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:45.289342 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 540/1067] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:45.295329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22260cd-0bdf-448e-b3d2-024a12c38c62 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:45.416339 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96126e8-0091-4410-9e89-d8f768a25225 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:45.417708 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:45.437645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:45.452206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22260cd-0bdf-448e-b3d2-024a12c38c62 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:45.465590 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5b8d7-d1c8-4f93-93c2-9601a0d8de67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:45.466235 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5b8d7-d1c8-4f93-93c2-9601a0d8de67 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:45.466519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5b8d7-d1c8-4f93-93c2-9601a0d8de67 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:45.466791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5b8d7-d1c8-4f93-93c2-9601a0d8de67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:45.518898 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daed783-210b-43a0-b8fa-3c27d77b3732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:45.519762 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daed783-210b-43a0-b8fa-3c27d77b3732 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:45.520856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daed783-210b-43a0-b8fa-3c27d77b3732 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:45.521332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daed783-210b-43a0-b8fa-3c27d77b3732 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:45.536003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daed783-210b-43a0-b8fa-3c27d77b3732 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:45.539350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daed783-210b-43a0-b8fa-3c27d77b3732 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:45.554162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-3daed783-210b-43a0-b8fa-3c27d77b3732 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:31:45.554162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:45.554162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:45.554162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:45.554162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:45.554162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:45.554162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:45.554162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:45.554162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:45.554162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:45.554162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:45.554162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:45.554162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:31:45.554162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:31:45.554162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:31:45.554162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:45.562646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:45.563601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:45.563601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:45.563601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:31:45.563601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:31:45.563601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:31:45.563601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:31:45.563601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:31:45.563601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:31:45.563601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:31:45.563601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:45.563601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 541/1068] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 26 05:31:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:45.564629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daed783-210b-43a0-b8fa-3c27d77b3732 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:45.567420 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5b8d7-d1c8-4f93-93c2-9601a0d8de67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:45.568966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5b8d7-d1c8-4f93-93c2-9601a0d8de67 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:45.689882 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 528/1069] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:31:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 05:31:45.696156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5b8d7-d1c8-4f93-93c2-9601a0d8de67 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:45.714841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:45.740307 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daed783-210b-43a0-b8fa-3c27d77b3732 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:45.789888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26090e95-ca0a-40d1-b01f-9633f951fc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:45.790442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26090e95-ca0a-40d1-b01f-9633f951fc3f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:45.790720 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26090e95-ca0a-40d1-b01f-9633f951fc3f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:45.790988 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26090e95-ca0a-40d1-b01f-9633f951fc3f None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:45.791775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26090e95-ca0a-40d1-b01f-9633f951fc3f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:45.792080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26090e95-ca0a-40d1-b01f-9633f951fc3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:45.793534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26090e95-ca0a-40d1-b01f-9633f951fc3f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:45.813194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5b8d7-d1c8-4f93-93c2-9601a0d8de67 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:45.816812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 542/1070] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 26 05:31:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:45.827043 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26090e95-ca0a-40d1-b01f-9633f951fc3f None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:45.831884 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:45.859331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15f1d7-5863-476c-a15d-7e0a29e7c923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:45.860096 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15f1d7-5863-476c-a15d-7e0a29e7c923 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:45.860379 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15f1d7-5863-476c-a15d-7e0a29e7c923 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:45.860664 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15f1d7-5863-476c-a15d-7e0a29e7c923 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:45.873006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15f1d7-5863-476c-a15d-7e0a29e7c923 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:45.874545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15f1d7-5863-476c-a15d-7e0a29e7c923 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:45.892999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-cb15f1d7-5863-476c-a15d-7e0a29e7c923 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:45.892999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:45.892999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:45.892999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:45.892999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:45.892999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:45.892999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:45.892999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:45.892999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:45.892999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:45.892999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:45.892999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:45.892999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:45.892999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:45.892999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:45.892999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:45.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:45.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:45.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:45.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:45.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:45.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:45.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:45.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:45.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:45.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:45.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:45.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:45.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:45.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:45.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:45.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:45.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:45.896719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:45.896719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:45.896719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:45.896719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 529/1071] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:45.900624 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15f1d7-5863-476c-a15d-7e0a29e7c923 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:45.968880 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26090e95-ca0a-40d1-b01f-9633f951fc3f None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:45.993285 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:46.014978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d98a7-99ef-42ba-97f3-ac1e78151d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:46.015796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d98a7-99ef-42ba-97f3-ac1e78151d53 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:46.016338 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d98a7-99ef-42ba-97f3-ac1e78151d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:46.016449 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456d98a7-99ef-42ba-97f3-ac1e78151d53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:46.028650 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d98a7-99ef-42ba-97f3-ac1e78151d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:46.030149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456d98a7-99ef-42ba-97f3-ac1e78151d53 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:46.046411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-456d98a7-99ef-42ba-97f3-ac1e78151d53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:46.046411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:46.046411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:46.046411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:46.046411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:46.046411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:46.046411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:46.046411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:46.046411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:46.046411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:46.046411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:46.046411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:46.046411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:46.046411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:46.046411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:46.046411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:46.047483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:46.047483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:46.047483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:46.047483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:46.047483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:46.047483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:46.047483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:46.047483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:46.047483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:46.047483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:46.047483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:46.047483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:46.047483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:46.047483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:46.047483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:46.047483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:46.047483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:46.054024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:46.054024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:46.054024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:46.054204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15f1d7-5863-476c-a15d-7e0a29e7c923 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:46.054292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 543/1072] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:46.054292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d98a7-99ef-42ba-97f3-ac1e78151d53 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:46.107021 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48e200-cbdb-49f8-b251-2aced351952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:46.107645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48e200-cbdb-49f8-b251-2aced351952f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:46.107919 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48e200-cbdb-49f8-b251-2aced351952f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:46.108187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48e200-cbdb-49f8-b251-2aced351952f None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:46.108839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48e200-cbdb-49f8-b251-2aced351952f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:46.109170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48e200-cbdb-49f8-b251-2aced351952f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:46.110545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48e200-cbdb-49f8-b251-2aced351952f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:46.124824 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 530/1073] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:46.150272 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:46.225930 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456d98a7-99ef-42ba-97f3-ac1e78151d53 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:46.283886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8de9a4-81cc-4bb9-812b-6899a0d637be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:46.284526 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8de9a4-81cc-4bb9-812b-6899a0d637be None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:46.284800 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8de9a4-81cc-4bb9-812b-6899a0d637be None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:46.285072 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8de9a4-81cc-4bb9-812b-6899a0d637be None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:46.285795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8de9a4-81cc-4bb9-812b-6899a0d637be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:46.286101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8de9a4-81cc-4bb9-812b-6899a0d637be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:46.287520 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8de9a4-81cc-4bb9-812b-6899a0d637be None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:46.304299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 544/1074] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:46.310980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48e200-cbdb-49f8-b251-2aced351952f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:46.480195 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48e200-cbdb-49f8-b251-2aced351952f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:46.514859 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:46.527720 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc42ec-d05e-4f2b-be34-1a6c6cde754f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:46.528801 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc42ec-d05e-4f2b-be34-1a6c6cde754f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:46.529341 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc42ec-d05e-4f2b-be34-1a6c6cde754f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:46.529825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc42ec-d05e-4f2b-be34-1a6c6cde754f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:46.541887 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc42ec-d05e-4f2b-be34-1a6c6cde754f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:46.544646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc42ec-d05e-4f2b-be34-1a6c6cde754f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:46.548065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a2a17-862d-4536-895b-34b60f05ffd0 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:46.548341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a2a17-862d-4536-895b-34b60f05ffd0 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:46.548613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a2a17-862d-4536-895b-34b60f05ffd0 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:46.550015 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 545/1075] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:46.555349 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094cce2-5926-4dd4-b730-697f1ab9a468 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:46.555565 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094cce2-5926-4dd4-b730-697f1ab9a468 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:46.555843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094cce2-5926-4dd4-b730-697f1ab9a468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:46.566094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-62bc42ec-d05e-4f2b-be34-1a6c6cde754f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:46.566094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:46.566094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:46.566094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:46.566094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:46.566094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:46.566094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:46.566094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:46.566094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:46.566094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:46.566094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:46.566094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:46.566094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:31:46.566094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:31:46.566094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:31:46.567075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:31:46.567075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:46.567075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:46.567075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:46.567075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:46.567075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:46.567075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:46.567075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:46.567075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:46.567075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:46.567075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:46.567075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:46.567075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:46.567075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:46.567075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:46.567075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:46.567075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:46.568434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:31:46.568434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:31:46.568434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:31:46.568434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:31:46.568434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:31:46.568434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:31:46.568434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:46.568434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:46.568434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 531/1076] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:31:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:46.574624 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-6094cce2-5926-4dd4-b730-697f1ab9a468 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:46.575618 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc42ec-d05e-4f2b-be34-1a6c6cde754f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:46.575828 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6094cce2-5926-4dd4-b730-697f1ab9a468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:46.650995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 546/1077] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:31:46.719437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc42ec-d05e-4f2b-be34-1a6c6cde754f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:46.757993 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57d4d4-fc63-416f-9f91-1d202512573e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:46.758602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57d4d4-fc63-416f-9f91-1d202512573e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:46.758877 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57d4d4-fc63-416f-9f91-1d202512573e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:46.759149 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57d4d4-fc63-416f-9f91-1d202512573e None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:46.759845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57d4d4-fc63-416f-9f91-1d202512573e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:46.760147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57d4d4-fc63-416f-9f91-1d202512573e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:46.761716 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57d4d4-fc63-416f-9f91-1d202512573e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:46.774883 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 532/1078] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:31:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:46.781221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094cce2-5926-4dd4-b730-697f1ab9a468 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:46.832116 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:46.894732 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094cce2-5926-4dd4-b730-697f1ab9a468 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:46.917973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57d4d4-fc63-416f-9f91-1d202512573e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:46.940877 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c013a-6a3f-4024-88a3-3f5a92176d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:46.941911 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c013a-6a3f-4024-88a3-3f5a92176d2f None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:46.942322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c013a-6a3f-4024-88a3-3f5a92176d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:46.942726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c013a-6a3f-4024-88a3-3f5a92176d2f None admin] PATH_INFO: `/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/92556048cbd04c409c3f4b6fd645627f/roles/aedef758940042499c7698d83130db60` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:46.946182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c013a-6a3f-4024-88a3-3f5a92176d2f None admin] RBAC: Authorizing `identity:create_grant(project_id=5e44d601292b47b7b66d6f3efa80b450, user_id=92556048cbd04c409c3f4b6fd645627f, role_id=aedef758940042499c7698d83130db60)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:46.948561 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c013a-6a3f-4024-88a3-3f5a92176d2f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:46.970672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 547/1079] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:31:46 2022] PUT /identity/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/92556048cbd04c409c3f4b6fd645627f/roles/aedef758940042499c7698d83130db60 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:31:46.977371 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:47.043600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57d4d4-fc63-416f-9f91-1d202512573e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:47.086598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b980de0-3e51-4591-aafb-5502c66dccbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:47.087165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b980de0-3e51-4591-aafb-5502c66dccbb None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:47.087447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b980de0-3e51-4591-aafb-5502c66dccbb None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:47.087773 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b980de0-3e51-4591-aafb-5502c66dccbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:47.101202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b980de0-3e51-4591-aafb-5502c66dccbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:31:47.106316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-9b980de0-3e51-4591-aafb-5502c66dccbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:31:47.106316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:47.106316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:47.106316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:47.106316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:47.106316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:47.106316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:47.106316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:47.106316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:47.106316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:47.106316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:47.106316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:47.106316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:31:47.106316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:31:47.106316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:31:47.106316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:31:47.108284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:31:47.108284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:31:47.108284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:31:47.108284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:31:47.108284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:47.108284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:47.108284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:47.108284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:47.108284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:47.108284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:47.108284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:47.108284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:47.108284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:47.108284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:47.108284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:47.108284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:47.110151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:47.110151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:47.110151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:47.110151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:47.110151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:31:47.110151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:31:47.110151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:31:47.110151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:31:47.110151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:31:47.110151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:31:47.110151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:31:47.110151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:47.113087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 533/1080] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:31:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:47.114330 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c013a-6a3f-4024-88a3-3f5a92176d2f None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:47.237634 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c013a-6a3f-4024-88a3-3f5a92176d2f None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:47.240190 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:47.277727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f9be1-4ff3-441a-a91f-a8d47a0b73c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:47.278350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f9be1-4ff3-441a-a91f-a8d47a0b73c6 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:47.278629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f9be1-4ff3-441a-a91f-a8d47a0b73c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:47.278896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f9be1-4ff3-441a-a91f-a8d47a0b73c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:47.279575 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f9be1-4ff3-441a-a91f-a8d47a0b73c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:47.279882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f9be1-4ff3-441a-a91f-a8d47a0b73c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:47.281284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f9be1-4ff3-441a-a91f-a8d47a0b73c6 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:47.290315 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 548/1081] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:31:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:47.296334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b980de0-3e51-4591-aafb-5502c66dccbb None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:47.401757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b980de0-3e51-4591-aafb-5502c66dccbb None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:47.411989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:47.439921 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dbcf3-4571-4aab-9902-a67a705b1d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:47.440559 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dbcf3-4571-4aab-9902-a67a705b1d73 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:47.440833 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dbcf3-4571-4aab-9902-a67a705b1d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:47.441102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dbcf3-4571-4aab-9902-a67a705b1d73 None admin] PATH_INFO: `/v3/users/291cf39fa0e4457c956911cc8bb06c08` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:47.442287 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8dbcf3-4571-4aab-9902-a67a705b1d73 None admin] RBAC: Authorizing `identity:get_user(user_id=291cf39fa0e4457c956911cc8bb06c08)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:47.443722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8dbcf3-4571-4aab-9902-a67a705b1d73 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:47.445374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 534/1082] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:31:47 2022] GET /identity/v3/users/291cf39fa0e4457c956911cc8bb06c08 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:47.451661 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f9be1-4ff3-441a-a91f-a8d47a0b73c6 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:47.560123 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f9be1-4ff3-441a-a91f-a8d47a0b73c6 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:47.593929 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e88e93-0977-4033-a1b1-bcc8a53025d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:47.594572 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e88e93-0977-4033-a1b1-bcc8a53025d0 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:47.594849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e88e93-0977-4033-a1b1-bcc8a53025d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:47.595116 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e88e93-0977-4033-a1b1-bcc8a53025d0 None admin] PATH_INFO: `/v3/projects/0cd9ee5a4a9247ddb49223aa91faad03` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:47.596670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e88e93-0977-4033-a1b1-bcc8a53025d0 None admin] RBAC: Authorizing `identity:get_project(project_id=0cd9ee5a4a9247ddb49223aa91faad03)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:47.598129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e88e93-0977-4033-a1b1-bcc8a53025d0 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:47.599778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 549/1083] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:31:47 2022] GET /identity/v3/projects/0cd9ee5a4a9247ddb49223aa91faad03 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:47.606116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dbcf3-4571-4aab-9902-a67a705b1d73 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:47.731857 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dbcf3-4571-4aab-9902-a67a705b1d73 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:47.772771 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2355909-71f1-4082-8d05-e52677b056a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:47.773473 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2355909-71f1-4082-8d05-e52677b056a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:47.773752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2355909-71f1-4082-8d05-e52677b056a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:47.774013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2355909-71f1-4082-8d05-e52677b056a8 None admin] PATH_INFO: `/v3/projects/0cd9ee5a4a9247ddb49223aa91faad03/users/291cf39fa0e4457c956911cc8bb06c08/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:47.775928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2355909-71f1-4082-8d05-e52677b056a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0cd9ee5a4a9247ddb49223aa91faad03, user_id=291cf39fa0e4457c956911cc8bb06c08, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:47.777463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2355909-71f1-4082-8d05-e52677b056a8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:47.791622 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 535/1084] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:31:47 2022] PUT /identity/v3/projects/0cd9ee5a4a9247ddb49223aa91faad03/users/291cf39fa0e4457c956911cc8bb06c08/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:31:49.195098 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb1cd8-610c-4230-912e-95c914a771a2 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:49.195808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb1cd8-610c-4230-912e-95c914a771a2 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:49.195808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb1cd8-610c-4230-912e-95c914a771a2 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:49.196996 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 550/1085] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:31:49.203823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d705cf0-8e2f-4833-a4be-de3ca745b114 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:49.204323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d705cf0-8e2f-4833-a4be-de3ca745b114 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:49.204553 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d705cf0-8e2f-4833-a4be-de3ca745b114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:49.234228 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7d705cf0-8e2f-4833-a4be-de3ca745b114 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:49.236177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7d705cf0-8e2f-4833-a4be-de3ca745b114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:49.271856 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:49.316012 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 536/1086] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:49.584056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb1cd8-610c-4230-912e-95c914a771a2 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:49.624455 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:49.700991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb1cd8-610c-4230-912e-95c914a771a2 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:49.736940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca74e6-3162-45fc-8c16-56d0d14820cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:49.737607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca74e6-3162-45fc-8c16-56d0d14820cd None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:49.737881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca74e6-3162-45fc-8c16-56d0d14820cd None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:49.738147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca74e6-3162-45fc-8c16-56d0d14820cd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:49.749082 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca74e6-3162-45fc-8c16-56d0d14820cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:31:49.753925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-a1ca74e6-3162-45fc-8c16-56d0d14820cd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:31:49.753925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:49.753925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:49.753925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:49.753925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:49.753925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:49.753925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:49.753925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:49.753925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:49.753925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:49.753925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:49.753925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:49.753925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:31:49.753925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:31:49.753925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:31:49.753925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:31:49.757614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:31:49.757614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:31:49.757614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:31:49.757614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:31:49.757614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:49.757614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:49.757614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:49.757614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:49.757614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:49.757614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:49.757614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:49.757614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:49.757614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:49.757614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:49.757614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:49.757614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:49.758496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:49.758496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:49.758496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:49.758496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:49.758496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:31:49.758496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:31:49.758496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:31:49.758496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:31:49.758496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:31:49.758496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:31:49.758496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 05:31:49.758496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:49.758496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 551/1087] 10.222.0.57 () {56 vars in 1120 bytes} [Mon Sep 26 05:31:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:49.766451 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d705cf0-8e2f-4833-a4be-de3ca745b114 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:49.891580 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d705cf0-8e2f-4833-a4be-de3ca745b114 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:49.932148 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be03a4-7c5f-4411-a7ed-d3d22533845f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:49.932887 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be03a4-7c5f-4411-a7ed-d3d22533845f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:49.933278 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be03a4-7c5f-4411-a7ed-d3d22533845f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:49.933597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be03a4-7c5f-4411-a7ed-d3d22533845f None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:49.934317 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be03a4-7c5f-4411-a7ed-d3d22533845f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:49.934673 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be03a4-7c5f-4411-a7ed-d3d22533845f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:49.936109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be03a4-7c5f-4411-a7ed-d3d22533845f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:49.945454 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 537/1088] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Sep 26 05:31:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:49.952327 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca74e6-3162-45fc-8c16-56d0d14820cd None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:49.972196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ca9e7-d1bd-4df6-bebd-789395631710 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:49.972507 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ca9e7-d1bd-4df6-bebd-789395631710 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:49.972785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ca9e7-d1bd-4df6-bebd-789395631710 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:49.974236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 538/1089] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:49.979869 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b3e24-12cd-4c9e-b3f6-e51822a38cad None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:49.980147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b3e24-12cd-4c9e-b3f6-e51822a38cad None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:49.980414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b3e24-12cd-4c9e-b3f6-e51822a38cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:49.998973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-387b3e24-12cd-4c9e-b3f6-e51822a38cad None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:50.000236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-387b3e24-12cd-4c9e-b3f6-e51822a38cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:50.025653 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:50.076171 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:50.076895 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 539/1090] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:31:50.084444 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca74e6-3162-45fc-8c16-56d0d14820cd None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:50.130371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005982d-49d4-45fc-9bd3-706ae46c99a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:50.131076 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005982d-49d4-45fc-9bd3-706ae46c99a6 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:50.131434 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005982d-49d4-45fc-9bd3-706ae46c99a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:50.131737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005982d-49d4-45fc-9bd3-706ae46c99a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:50.143497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005982d-49d4-45fc-9bd3-706ae46c99a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:50.144916 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005982d-49d4-45fc-9bd3-706ae46c99a6 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:50.160892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-1005982d-49d4-45fc-9bd3-706ae46c99a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:50.160892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:50.160892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:50.160892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:50.160892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:50.160892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:50.160892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:50.160892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:50.160892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:50.160892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:50.160892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:50.160892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:50.160892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:50.160892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:50.160892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:50.160892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:50.162404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:50.162404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:50.162404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:50.162404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:50.162404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:50.162404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:50.162404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:50.162404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:50.162404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:50.162404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:50.162404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:50.162404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:50.162404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:50.162404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:50.162404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:50.162404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:50.162404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:50.163448 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:50.163448 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:50.163448 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:50.165335 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 552/1091] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:50.168170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b3e24-12cd-4c9e-b3f6-e51822a38cad None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:50.290638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b3e24-12cd-4c9e-b3f6-e51822a38cad None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:50.340160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f159c68-defd-4e1b-b396-96ca83cb5a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:50.340798 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f159c68-defd-4e1b-b396-96ca83cb5a86 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:50.341078 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f159c68-defd-4e1b-b396-96ca83cb5a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:50.341369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f159c68-defd-4e1b-b396-96ca83cb5a86 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:50.342044 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f159c68-defd-4e1b-b396-96ca83cb5a86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:50.342348 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f159c68-defd-4e1b-b396-96ca83cb5a86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:50.343770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f159c68-defd-4e1b-b396-96ca83cb5a86 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:50.345248 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005982d-49d4-45fc-9bd3-706ae46c99a6 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:50.358118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 540/1092] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:50.364353 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f159c68-defd-4e1b-b396-96ca83cb5a86 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:50.386184 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:50.473820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005982d-49d4-45fc-9bd3-706ae46c99a6 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:50.489006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f159c68-defd-4e1b-b396-96ca83cb5a86 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:50.514771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095295f9-953e-4d4d-830a-62c5386330eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:50.515420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095295f9-953e-4d4d-830a-62c5386330eb None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:50.515700 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095295f9-953e-4d4d-830a-62c5386330eb None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:50.515977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095295f9-953e-4d4d-830a-62c5386330eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:50.525256 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5b8f9-7c17-4c18-901a-52319fa1a55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:50.525907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5b8f9-7c17-4c18-901a-52319fa1a55c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:50.526208 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5b8f9-7c17-4c18-901a-52319fa1a55c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:50.526480 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5b8f9-7c17-4c18-901a-52319fa1a55c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:50.531049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095295f9-953e-4d4d-830a-62c5386330eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:31:50.535072 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5b8f9-7c17-4c18-901a-52319fa1a55c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:50.536533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5b8f9-7c17-4c18-901a-52319fa1a55c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:50.538663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-095295f9-953e-4d4d-830a-62c5386330eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:31:50.538663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:50.538663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:50.538663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:50.538663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:50.538663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:50.538663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:50.538663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:50.538663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:50.538663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:50.538663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:50.538663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:50.538663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:31:50.538663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:31:50.538663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:31:50.538663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:31:50.539698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:31:50.539698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:31:50.539698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:31:50.539698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:31:50.539698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:50.539698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:50.539698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:50.539698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:50.539698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:50.539698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:50.539698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:50.539698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:50.539698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:50.539698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:50.539698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:50.539698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:50.542071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:50.542071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:50.542071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:50.542071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:50.542071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:31:50.542071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:31:50.542071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:31:50.542071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:31:50.542071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:31:50.542071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:31:50.542071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:31:50.542071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:31:50.542071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 553/1093] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:31:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:50.547194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095295f9-953e-4d4d-830a-62c5386330eb None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:50.549405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-23d5b8f9-7c17-4c18-901a-52319fa1a55c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:31:50.549405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:50.549405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:50.549405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:50.549405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:50.549405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:50.549405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:50.549405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:50.549405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:50.549405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:50.549405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:50.549405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:50.549405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:31:50.549405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:31:50.549405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:31:50.549405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:50.550402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:50.551373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:50.551373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:50.551373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:31:50.551373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:31:50.551373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:31:50.551373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:31:50.551373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:31:50.551373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:31:50.551373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 05:31:50.551373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:50.551373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 541/1094] 10.222.0.57 () {56 vars in 1093 bytes} [Mon Sep 26 05:31:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:50.556563 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5b8f9-7c17-4c18-901a-52319fa1a55c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:50.561639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:50.640967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:50.664361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5b8f9-7c17-4c18-901a-52319fa1a55c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:50.667749 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095295f9-953e-4d4d-830a-62c5386330eb None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:50.702387 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5aac8-7feb-4ebc-aafb-f8ff26648e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:50.703020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5aac8-7feb-4ebc-aafb-f8ff26648e1f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:50.703186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79572f17-86ed-4e87-bac9-ecc26ddee437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:50.703372 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5aac8-7feb-4ebc-aafb-f8ff26648e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:50.703502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5aac8-7feb-4ebc-aafb-f8ff26648e1f None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:50.703666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79572f17-86ed-4e87-bac9-ecc26ddee437 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:50.703969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79572f17-86ed-4e87-bac9-ecc26ddee437 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:50.704238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79572f17-86ed-4e87-bac9-ecc26ddee437 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:50.704500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5aac8-7feb-4ebc-aafb-f8ff26648e1f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:50.704635 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5aac8-7feb-4ebc-aafb-f8ff26648e1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:50.704906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79572f17-86ed-4e87-bac9-ecc26ddee437 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:50.705211 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79572f17-86ed-4e87-bac9-ecc26ddee437 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:50.706015 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5aac8-7feb-4ebc-aafb-f8ff26648e1f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:50.706581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79572f17-86ed-4e87-bac9-ecc26ddee437 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:50.716782 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 554/1095] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:31:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:50.723551 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79572f17-86ed-4e87-bac9-ecc26ddee437 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:50.726238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 542/1096] 10.222.0.57 () {56 vars in 1133 bytes} [Mon Sep 26 05:31:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:50.732300 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5aac8-7feb-4ebc-aafb-f8ff26648e1f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:50.799513 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:50.849893 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79572f17-86ed-4e87-bac9-ecc26ddee437 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:50.850116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5aac8-7feb-4ebc-aafb-f8ff26648e1f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:50.888765 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4240b45-0531-45a4-aef1-01b760811008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:50.889427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4240b45-0531-45a4-aef1-01b760811008 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:50.889566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c174833-2219-4777-974a-97a570e5b007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:50.889732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4240b45-0531-45a4-aef1-01b760811008 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:50.889967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4240b45-0531-45a4-aef1-01b760811008 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:50.890293 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c174833-2219-4777-974a-97a570e5b007 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:50.890576 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c174833-2219-4777-974a-97a570e5b007 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:50.891033 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c174833-2219-4777-974a-97a570e5b007 None admin] PATH_INFO: `/v3/users/291cf39fa0e4457c956911cc8bb06c08` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:50.892357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c174833-2219-4777-974a-97a570e5b007 None admin] RBAC: Authorizing `identity:get_user(user_id=291cf39fa0e4457c956911cc8bb06c08)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:50.893837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c174833-2219-4777-974a-97a570e5b007 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:50.894763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:50.896138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 555/1097] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:31:50 2022] GET /identity/v3/users/291cf39fa0e4457c956911cc8bb06c08 => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:50.902470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c174833-2219-4777-974a-97a570e5b007 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:50.982471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4240b45-0531-45a4-aef1-01b760811008 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:50.983880 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4240b45-0531-45a4-aef1-01b760811008 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:51.015289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c174833-2219-4777-974a-97a570e5b007 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:51.060842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d6bed-14f8-4698-8d21-074d37722dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:51.061492 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d6bed-14f8-4698-8d21-074d37722dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:51.061766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d6bed-14f8-4698-8d21-074d37722dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:51.062038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d6bed-14f8-4698-8d21-074d37722dc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:51.066907 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:51.101184 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 543/1098] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:31:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 05:31:51.108760 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4240b45-0531-45a4-aef1-01b760811008 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:51.148010 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:51.153221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d6bed-14f8-4698-8d21-074d37722dc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:51.155507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d6bed-14f8-4698-8d21-074d37722dc7 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:51.220650 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4240b45-0531-45a4-aef1-01b760811008 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:51.261257 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302991af-a2cf-4cbc-bd49-574896e09160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:51.261904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302991af-a2cf-4cbc-bd49-574896e09160 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:51.262179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302991af-a2cf-4cbc-bd49-574896e09160 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:51.262445 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302991af-a2cf-4cbc-bd49-574896e09160 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:51.270773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 556/1099] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:31:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 05:31:51.273467 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302991af-a2cf-4cbc-bd49-574896e09160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:51.274933 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302991af-a2cf-4cbc-bd49-574896e09160 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:51.277559 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d6bed-14f8-4698-8d21-074d37722dc7 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:51.291560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-302991af-a2cf-4cbc-bd49-574896e09160 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:51.291560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:51.291560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:51.291560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:51.291560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:51.291560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:51.291560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:51.291560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:51.291560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:51.291560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:51.291560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:51.291560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:51.291560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:31:51.291560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:31:51.291560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:31:51.291560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:31:51.293460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:51.293460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:51.293460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:51.293460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:51.293460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:51.293460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:51.293460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:51.293460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:51.293460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:51.293460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:51.293460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:51.293460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:51.293460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:51.293460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:51.293460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:51.293460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:51.293460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:31:51.294872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:31:51.294872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:31:51.294872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:51.294872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 544/1100] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:31:51.298797 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302991af-a2cf-4cbc-bd49-574896e09160 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:51.317504 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:51.391009 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d6bed-14f8-4698-8d21-074d37722dc7 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:51.403825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:51.418500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302991af-a2cf-4cbc-bd49-574896e09160 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:51.442101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad2e09-21b1-4089-b1dd-72dfdbfc8c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:51.442759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad2e09-21b1-4089-b1dd-72dfdbfc8c8c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:51.443032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad2e09-21b1-4089-b1dd-72dfdbfc8c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:51.443306 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad2e09-21b1-4089-b1dd-72dfdbfc8c8c None admin] PATH_INFO: `/v3/projects/0cd9ee5a4a9247ddb49223aa91faad03` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:51.444505 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad2e09-21b1-4089-b1dd-72dfdbfc8c8c None admin] RBAC: Authorizing `identity:get_project(project_id=0cd9ee5a4a9247ddb49223aa91faad03)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:51.445980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad2e09-21b1-4089-b1dd-72dfdbfc8c8c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:51.448257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 557/1101] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:31:51 2022] GET /identity/v3/projects/0cd9ee5a4a9247ddb49223aa91faad03 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:51.454866 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad2e09-21b1-4089-b1dd-72dfdbfc8c8c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:51.459495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7a2b5-4df4-44e2-8615-37437b178f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:51.460540 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7a2b5-4df4-44e2-8615-37437b178f90 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:51.460923 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7a2b5-4df4-44e2-8615-37437b178f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:51.461328 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7a2b5-4df4-44e2-8615-37437b178f90 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:51.462065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7a2b5-4df4-44e2-8615-37437b178f90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:51.462428 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7a2b5-4df4-44e2-8615-37437b178f90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:51.464075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7a2b5-4df4-44e2-8615-37437b178f90 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:51.480523 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 545/1102] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:51.486975 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7a2b5-4df4-44e2-8615-37437b178f90 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:51.592895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad2e09-21b1-4089-b1dd-72dfdbfc8c8c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:51.595819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:51.606441 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7a2b5-4df4-44e2-8615-37437b178f90 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:51.632362 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd7d23-541f-4ad4-a87d-ab5b88d2bc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:51.633149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd7d23-541f-4ad4-a87d-ab5b88d2bc18 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:51.633591 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd7d23-541f-4ad4-a87d-ab5b88d2bc18 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:51.633950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd7d23-541f-4ad4-a87d-ab5b88d2bc18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:51.634979 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd7d23-541f-4ad4-a87d-ab5b88d2bc18 None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=0cd9ee5a4a9247ddb49223aa91faad03, user.id=291cf39fa0e4457c956911cc8bb06c08) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:51.635510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd7d23-541f-4ad4-a87d-ab5b88d2bc18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:51.636994 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd7d23-541f-4ad4-a87d-ab5b88d2bc18 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:51.643538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 558/1103] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:31:51 2022] GET /identity/v3/role_assignments?user.id=291cf39fa0e4457c956911cc8bb06c08&scope.project.id=0cd9ee5a4a9247ddb49223aa91faad03&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:51.643991 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55e7e5-9b03-4db4-9945-0e21dc49aca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:51.644620 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55e7e5-9b03-4db4-9945-0e21dc49aca7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:51.644890 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55e7e5-9b03-4db4-9945-0e21dc49aca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:51.645403 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55e7e5-9b03-4db4-9945-0e21dc49aca7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:51.657824 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55e7e5-9b03-4db4-9945-0e21dc49aca7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:51.660538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55e7e5-9b03-4db4-9945-0e21dc49aca7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:51.682565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-4a55e7e5-9b03-4db4-9945-0e21dc49aca7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:51.682565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:31:51.682565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:31:51.682565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:31:51.682565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:31:51.682565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:31:51.682565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:31:51.682565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:31:51.682565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:31:51.682565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:31:51.682565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:31:51.682565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:31:51.682565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:31:51.682565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:31:51.682565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:31:51.683973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:31:51.683973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:31:51.683973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:31:51.683973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:31:51.683973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:31:51.683973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:31:51.683973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:31:51.683973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:31:51.683973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:31:51.683973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:31:51.683973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:31:51.683973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:31:51.683973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:31:51.683973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:31:51.683973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:31:51.683973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:31:51.683973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:31:51.685645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:31:51.685645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:31:51.685645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:31:51.685645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:31:51.685645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:31:51.685645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:31:51.685645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:31:51.685645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:31:51.685645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 546/1104] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:31:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:31:51.691123 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd7d23-541f-4ad4-a87d-ab5b88d2bc18 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:51.821860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd7d23-541f-4ad4-a87d-ab5b88d2bc18 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:51.866761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c179e-953c-4949-bdc6-a67bb9e14f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:51.867406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c179e-953c-4949-bdc6-a67bb9e14f2b None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:51.867680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c179e-953c-4949-bdc6-a67bb9e14f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:51.867950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c179e-953c-4949-bdc6-a67bb9e14f2b None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:51.868668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c179e-953c-4949-bdc6-a67bb9e14f2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:51.868976 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c179e-953c-4949-bdc6-a67bb9e14f2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:51.870591 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c179e-953c-4949-bdc6-a67bb9e14f2b None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:51.887541 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 559/1105] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:31:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:31:51.895820 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55e7e5-9b03-4db4-9945-0e21dc49aca7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:51.910680 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:52.025734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55e7e5-9b03-4db4-9945-0e21dc49aca7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:52.067097 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d63768-30ec-4e3f-a77c-7a179b184685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:52.067728 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d63768-30ec-4e3f-a77c-7a179b184685 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:52.067998 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d63768-30ec-4e3f-a77c-7a179b184685 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:52.068264 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d63768-30ec-4e3f-a77c-7a179b184685 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:52.069013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d63768-30ec-4e3f-a77c-7a179b184685 None admin] RBAC: Adding query filter params (role.id=aedef758940042499c7698d83130db60, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=92556048cbd04c409c3f4b6fd645627f) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:52.069352 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d63768-30ec-4e3f-a77c-7a179b184685 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:52.070844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d63768-30ec-4e3f-a77c-7a179b184685 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:52.078094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 547/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:31:51 2022] GET /identity/v3/role_assignments?user.id=92556048cbd04c409c3f4b6fd645627f&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=aedef758940042499c7698d83130db60 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:53.945915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd14356-edb0-4ea1-a362-e9bb4e0a7a1e None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:53.946521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd14356-edb0-4ea1-a362-e9bb4e0a7a1e None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:53.946521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd14356-edb0-4ea1-a362-e9bb4e0a7a1e None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:53.947772 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 560/1107] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:53.953916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671df83-cd87-4181-a1a6-b6fd8ac6cd6e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:53.954398 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671df83-cd87-4181-a1a6-b6fd8ac6cd6e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:53.954518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671df83-cd87-4181-a1a6-b6fd8ac6cd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:53.974908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7671df83-cd87-4181-a1a6-b6fd8ac6cd6e None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:53.976125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7671df83-cd87-4181-a1a6-b6fd8ac6cd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:54.062777 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 548/1108] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:54.331559 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd14356-edb0-4ea1-a362-e9bb4e0a7a1e None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:54.369571 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:54.445335 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd14356-edb0-4ea1-a362-e9bb4e0a7a1e None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:54.481289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a87dce-39d5-4647-aad1-2b35494019e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:54.481918 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a87dce-39d5-4647-aad1-2b35494019e9 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:54.482190 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a87dce-39d5-4647-aad1-2b35494019e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:54.482452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a87dce-39d5-4647-aad1-2b35494019e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:54.483647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a87dce-39d5-4647-aad1-2b35494019e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:54.485004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a87dce-39d5-4647-aad1-2b35494019e9 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:54.486617 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 561/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:31:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:54.492866 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671df83-cd87-4181-a1a6-b6fd8ac6cd6e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:54.495360 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:54.614341 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671df83-cd87-4181-a1a6-b6fd8ac6cd6e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:54.670077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c1645-7d6b-4d1b-9cd3-73608c1c49b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:54.671087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c1645-7d6b-4d1b-9cd3-73608c1c49b1 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:54.671509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c1645-7d6b-4d1b-9cd3-73608c1c49b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:54.671800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c1645-7d6b-4d1b-9cd3-73608c1c49b1 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:54.672496 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c1645-7d6b-4d1b-9cd3-73608c1c49b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:54.674396 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c1645-7d6b-4d1b-9cd3-73608c1c49b1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:54.698922 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 549/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:31:54 2022] POST /identity/v3/users => generated 286 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:31:56.943138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bc023-68cd-4f89-a3df-8bfc9c1a139f None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:56.943742 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bc023-68cd-4f89-a3df-8bfc9c1a139f None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:56.943742 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bc023-68cd-4f89-a3df-8bfc9c1a139f None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:56.944978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 562/1111] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:31:56.950907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad3716-e934-46d4-bbb1-f25e689e5640 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:56.951364 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad3716-e934-46d4-bbb1-f25e689e5640 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:56.951495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad3716-e934-46d4-bbb1-f25e689e5640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:56.971371 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-dbad3716-e934-46d4-bbb1-f25e689e5640 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:31:56.972599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-dbad3716-e934-46d4-bbb1-f25e689e5640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:31:57.029112 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:57.045062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 550/1112] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:31:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:31:57.310916 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bc023-68cd-4f89-a3df-8bfc9c1a139f None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:57.420157 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:57.428403 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bc023-68cd-4f89-a3df-8bfc9c1a139f None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:57.465443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d21fd-3f96-4521-86b2-341cc41a1b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:57.466078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d21fd-3f96-4521-86b2-341cc41a1b38 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:57.466357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d21fd-3f96-4521-86b2-341cc41a1b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:57.466629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d21fd-3f96-4521-86b2-341cc41a1b38 None admin] PATH_INFO: `/v3/roles/05e78a4a009c41488cf3d2c47e8cbe30` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:57.468057 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d21fd-3f96-4521-86b2-341cc41a1b38 None admin] RBAC: Authorizing `identity:get_role(role_id=05e78a4a009c41488cf3d2c47e8cbe30)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:57.469456 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d21fd-3f96-4521-86b2-341cc41a1b38 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:57.470814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 563/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:31:57 2022] GET /identity/v3/roles/05e78a4a009c41488cf3d2c47e8cbe30 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:57.477791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad3716-e934-46d4-bbb1-f25e689e5640 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:57.584639 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad3716-e934-46d4-bbb1-f25e689e5640 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:57.622942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b7492-8456-47bb-b8a1-59aa61daff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:57.623581 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b7492-8456-47bb-b8a1-59aa61daff49 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:57.623857 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b7492-8456-47bb-b8a1-59aa61daff49 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:57.624117 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b7492-8456-47bb-b8a1-59aa61daff49 None admin] PATH_INFO: `/v3/users/1596819a7b8b4ae5aaffd892cf8b8db3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:57.638923 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b7492-8456-47bb-b8a1-59aa61daff49 None admin] RBAC: Authorizing `identity:get_user(user_id=1596819a7b8b4ae5aaffd892cf8b8db3)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:57.640360 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b7492-8456-47bb-b8a1-59aa61daff49 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:57.642099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 551/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:31:57 2022] GET /identity/v3/users/1596819a7b8b4ae5aaffd892cf8b8db3 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:57.648320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d21fd-3f96-4521-86b2-341cc41a1b38 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:57.791505 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d21fd-3f96-4521-86b2-341cc41a1b38 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:57.838805 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c02f8-b642-4b0b-b1b3-a1690864cf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:57.839345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c02f8-b642-4b0b-b1b3-a1690864cf6a None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:57.839624 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c02f8-b642-4b0b-b1b3-a1690864cf6a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:57.839899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c02f8-b642-4b0b-b1b3-a1690864cf6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:57.872934 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:57.913900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c02f8-b642-4b0b-b1b3-a1690864cf6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:57.915361 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c02f8-b642-4b0b-b1b3-a1690864cf6a None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:58.016604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 564/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:31:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:31:58.023027 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b7492-8456-47bb-b8a1-59aa61daff49 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:58.071785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:58.127725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b7492-8456-47bb-b8a1-59aa61daff49 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:58.163786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cea78c-3653-4f43-aad3-99f84a2b5319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:58.164423 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cea78c-3653-4f43-aad3-99f84a2b5319 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:58.164703 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cea78c-3653-4f43-aad3-99f84a2b5319 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:58.164978 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cea78c-3653-4f43-aad3-99f84a2b5319 None admin] PATH_INFO: `/v3/projects/0cd9ee5a4a9247ddb49223aa91faad03` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:58.166157 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cea78c-3653-4f43-aad3-99f84a2b5319 None admin] RBAC: Authorizing `identity:get_project(project_id=0cd9ee5a4a9247ddb49223aa91faad03)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:58.167677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cea78c-3653-4f43-aad3-99f84a2b5319 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:58.169384 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 552/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:31:58 2022] GET /identity/v3/projects/0cd9ee5a4a9247ddb49223aa91faad03 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:31:58.175791 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c02f8-b642-4b0b-b1b3-a1690864cf6a None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:31:58.241659 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:31:58.285220 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c02f8-b642-4b0b-b1b3-a1690864cf6a None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:31:58.323836 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd7dfe-abc7-4e5c-a45c-57be1c12c58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:31:58.324442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd7dfe-abc7-4e5c-a45c-57be1c12c58a None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:31:58.324717 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd7dfe-abc7-4e5c-a45c-57be1c12c58a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:31:58.324983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd7dfe-abc7-4e5c-a45c-57be1c12c58a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:31:58.325737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd7dfe-abc7-4e5c-a45c-57be1c12c58a None admin] RBAC: Adding query filter params (role.id=05e78a4a009c41488cf3d2c47e8cbe30, scope.project.id=0cd9ee5a4a9247ddb49223aa91faad03, user.id=1596819a7b8b4ae5aaffd892cf8b8db3) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:31:58.326041 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd7dfe-abc7-4e5c-a45c-57be1c12c58a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:31:58.327469 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd7dfe-abc7-4e5c-a45c-57be1c12c58a None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:31:58.333100 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 565/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:31:58 2022] GET /identity/v3/role_assignments?user.id=1596819a7b8b4ae5aaffd892cf8b8db3&scope.project.id=0cd9ee5a4a9247ddb49223aa91faad03&role.id=05e78a4a009c41488cf3d2c47e8cbe30 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:00.543888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd509267-7ed0-4163-b434-0e24e8298291 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:00.544435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd509267-7ed0-4163-b434-0e24e8298291 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:00.544435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd509267-7ed0-4163-b434-0e24e8298291 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:00.545729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 553/1118] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:32:00.551813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4ed42-44b4-4a29-9723-5db284a76e44 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:00.552361 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4ed42-44b4-4a29-9723-5db284a76e44 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:00.552361 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4ed42-44b4-4a29-9723-5db284a76e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:00.573642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d8b4ed42-44b4-4a29-9723-5db284a76e44 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:00.575324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d8b4ed42-44b4-4a29-9723-5db284a76e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:00.649239 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 566/1119] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:32:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:32:00.921350 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd509267-7ed0-4163-b434-0e24e8298291 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:01.028434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd509267-7ed0-4163-b434-0e24e8298291 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:01.043559 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:01.066050 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18b298-c536-433e-aa1b-a9d62d7348b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:01.066681 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18b298-c536-433e-aa1b-a9d62d7348b7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:01.066953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18b298-c536-433e-aa1b-a9d62d7348b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:01.067217 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18b298-c536-433e-aa1b-a9d62d7348b7 None admin] PATH_INFO: `/v3/roles/05e78a4a009c41488cf3d2c47e8cbe30` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:01.068654 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18b298-c536-433e-aa1b-a9d62d7348b7 None admin] RBAC: Authorizing `identity:get_role(role_id=05e78a4a009c41488cf3d2c47e8cbe30)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:01.070069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18b298-c536-433e-aa1b-a9d62d7348b7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:01.071407 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 554/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:32:00 2022] GET /identity/v3/roles/05e78a4a009c41488cf3d2c47e8cbe30 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:01.077842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4ed42-44b4-4a29-9723-5db284a76e44 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:01.139913 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:01.189516 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4ed42-44b4-4a29-9723-5db284a76e44 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:01.229452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af37c7-e8b0-49d3-a565-4af98c8eec8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:01.230105 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af37c7-e8b0-49d3-a565-4af98c8eec8e None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:01.230387 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af37c7-e8b0-49d3-a565-4af98c8eec8e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:01.230662 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af37c7-e8b0-49d3-a565-4af98c8eec8e None admin] PATH_INFO: `/v3/users/1596819a7b8b4ae5aaffd892cf8b8db3` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:01.231862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af37c7-e8b0-49d3-a565-4af98c8eec8e None admin] RBAC: Authorizing `identity:get_user(user_id=1596819a7b8b4ae5aaffd892cf8b8db3)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:01.233312 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af37c7-e8b0-49d3-a565-4af98c8eec8e None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:01.234927 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 567/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:32:01 2022] GET /identity/v3/users/1596819a7b8b4ae5aaffd892cf8b8db3 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:01.241076 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18b298-c536-433e-aa1b-a9d62d7348b7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:01.347932 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18b298-c536-433e-aa1b-a9d62d7348b7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:01.384901 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40b7de-b93d-4e43-abf1-3365cdcfd0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:01.385596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40b7de-b93d-4e43-abf1-3365cdcfd0ee None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:01.385853 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40b7de-b93d-4e43-abf1-3365cdcfd0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:01.386120 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40b7de-b93d-4e43-abf1-3365cdcfd0ee None admin] PATH_INFO: `/v3/projects/0cd9ee5a4a9247ddb49223aa91faad03` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:01.387238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40b7de-b93d-4e43-abf1-3365cdcfd0ee None admin] RBAC: Authorizing `identity:get_project(project_id=0cd9ee5a4a9247ddb49223aa91faad03)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:01.388671 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40b7de-b93d-4e43-abf1-3365cdcfd0ee None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:01.390355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 555/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:32:01 2022] GET /identity/v3/projects/0cd9ee5a4a9247ddb49223aa91faad03 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:01.396717 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af37c7-e8b0-49d3-a565-4af98c8eec8e None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:01.501456 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af37c7-e8b0-49d3-a565-4af98c8eec8e None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:01.527355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:01.536140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e69f28-34a1-4790-8edd-2ed03bfba323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:01.536801 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e69f28-34a1-4790-8edd-2ed03bfba323 None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:01.537079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e69f28-34a1-4790-8edd-2ed03bfba323 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:01.537378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e69f28-34a1-4790-8edd-2ed03bfba323 None admin] PATH_INFO: `/v3/projects/0cd9ee5a4a9247ddb49223aa91faad03/users/1596819a7b8b4ae5aaffd892cf8b8db3/roles/05e78a4a009c41488cf3d2c47e8cbe30` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:01.539550 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e69f28-34a1-4790-8edd-2ed03bfba323 None admin] RBAC: Authorizing `identity:create_grant(project_id=0cd9ee5a4a9247ddb49223aa91faad03, user_id=1596819a7b8b4ae5aaffd892cf8b8db3, role_id=05e78a4a009c41488cf3d2c47e8cbe30)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:01.540982 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e69f28-34a1-4790-8edd-2ed03bfba323 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:01.554741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 568/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:32:01 2022] PUT /identity/v3/projects/0cd9ee5a4a9247ddb49223aa91faad03/users/1596819a7b8b4ae5aaffd892cf8b8db3/roles/05e78a4a009c41488cf3d2c47e8cbe30 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:32:03.681166 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2008f-75cb-4772-840c-48b077e760cc None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:03.681733 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2008f-75cb-4772-840c-48b077e760cc None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:03.681733 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2008f-75cb-4772-840c-48b077e760cc None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:03.683183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 556/1124] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:32:03.688839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219210-a1d3-4aaa-83cb-658c48132972 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:03.689358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219210-a1d3-4aaa-83cb-658c48132972 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:03.689463 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219210-a1d3-4aaa-83cb-658c48132972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:03.708285 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-fb219210-a1d3-4aaa-83cb-658c48132972 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:03.709514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-fb219210-a1d3-4aaa-83cb-658c48132972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:03.781345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 569/1125] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:32:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:32:04.045405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2008f-75cb-4772-840c-48b077e760cc None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:04.102781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:04.158333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2008f-75cb-4772-840c-48b077e760cc None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:04.196760 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8455a-3f5c-4d72-812d-ea4a99e45c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:04.197503 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8455a-3f5c-4d72-812d-ea4a99e45c86 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:04.197796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8455a-3f5c-4d72-812d-ea4a99e45c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:04.198020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8455a-3f5c-4d72-812d-ea4a99e45c86 None admin] PATH_INFO: `/v3/roles/05e78a4a009c41488cf3d2c47e8cbe30` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:04.199464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8455a-3f5c-4d72-812d-ea4a99e45c86 None admin] RBAC: Authorizing `identity:get_role(role_id=05e78a4a009c41488cf3d2c47e8cbe30)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:04.200855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8455a-3f5c-4d72-812d-ea4a99e45c86 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:04.202265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 557/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:32:04 2022] GET /identity/v3/roles/05e78a4a009c41488cf3d2c47e8cbe30 => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:04.208503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb219210-a1d3-4aaa-83cb-658c48132972 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:04.312878 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb219210-a1d3-4aaa-83cb-658c48132972 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:04.329756 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:04.349791 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810df07-d4db-41a7-b0ee-dbe2d2844503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:04.350401 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810df07-d4db-41a7-b0ee-dbe2d2844503 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:04.350675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810df07-d4db-41a7-b0ee-dbe2d2844503 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:04.350969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810df07-d4db-41a7-b0ee-dbe2d2844503 None admin] PATH_INFO: `/v3/users/1596819a7b8b4ae5aaffd892cf8b8db3` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:04.352083 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810df07-d4db-41a7-b0ee-dbe2d2844503 None admin] RBAC: Authorizing `identity:get_user(user_id=1596819a7b8b4ae5aaffd892cf8b8db3)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:04.353508 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810df07-d4db-41a7-b0ee-dbe2d2844503 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:04.355139 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 570/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:32:04 2022] GET /identity/v3/users/1596819a7b8b4ae5aaffd892cf8b8db3 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:04.361603 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8455a-3f5c-4d72-812d-ea4a99e45c86 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:04.466314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8455a-3f5c-4d72-812d-ea4a99e45c86 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:04.484990 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:04.504705 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e25b0-ee89-4d3f-8243-5c8996b455a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:04.505391 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e25b0-ee89-4d3f-8243-5c8996b455a0 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:04.505649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e25b0-ee89-4d3f-8243-5c8996b455a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:04.505919 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e25b0-ee89-4d3f-8243-5c8996b455a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:04.578699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e25b0-ee89-4d3f-8243-5c8996b455a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:04.580052 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e25b0-ee89-4d3f-8243-5c8996b455a0 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:04.686946 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 558/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:32:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:32:04.693373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810df07-d4db-41a7-b0ee-dbe2d2844503 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:04.802000 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810df07-d4db-41a7-b0ee-dbe2d2844503 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:04.836112 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9cace-3f48-47d1-981e-fb9bc40c24af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:04.836653 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9cace-3f48-47d1-981e-fb9bc40c24af None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:04.836932 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9cace-3f48-47d1-981e-fb9bc40c24af None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:04.837236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9cace-3f48-47d1-981e-fb9bc40c24af None admin] PATH_INFO: `/v3/projects/0cd9ee5a4a9247ddb49223aa91faad03` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:04.838423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9cace-3f48-47d1-981e-fb9bc40c24af None admin] RBAC: Authorizing `identity:get_project(project_id=0cd9ee5a4a9247ddb49223aa91faad03)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:04.839887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9cace-3f48-47d1-981e-fb9bc40c24af None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:04.841562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 571/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:32:04 2022] GET /identity/v3/projects/0cd9ee5a4a9247ddb49223aa91faad03 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:32:04.848617 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e25b0-ee89-4d3f-8243-5c8996b455a0 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:04.869361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:04.952428 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e25b0-ee89-4d3f-8243-5c8996b455a0 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:04.989513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2779d-443f-47b4-9aba-55600d042cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:04.990124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2779d-443f-47b4-9aba-55600d042cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:04.990404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2779d-443f-47b4-9aba-55600d042cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:04.990678 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2779d-443f-47b4-9aba-55600d042cb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:04.991413 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2779d-443f-47b4-9aba-55600d042cb3 None admin] RBAC: Adding query filter params (role.id=05e78a4a009c41488cf3d2c47e8cbe30, scope.project.id=0cd9ee5a4a9247ddb49223aa91faad03, user.id=1596819a7b8b4ae5aaffd892cf8b8db3) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:32:04.991725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2779d-443f-47b4-9aba-55600d042cb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:04.993178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2779d-443f-47b4-9aba-55600d042cb3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:04.999503 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 559/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:32:04 2022] GET /identity/v3/role_assignments?user.id=1596819a7b8b4ae5aaffd892cf8b8db3&scope.project.id=0cd9ee5a4a9247ddb49223aa91faad03&role.id=05e78a4a009c41488cf3d2c47e8cbe30 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:07.195659 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec36a19-16e9-4302-bedd-357c9b938a65 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:07.196281 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec36a19-16e9-4302-bedd-357c9b938a65 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:07.196281 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec36a19-16e9-4302-bedd-357c9b938a65 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:07.197593 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 572/1131] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:32:07.203303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27142ef4-c7f7-42fe-ae50-d8a4cfb0753a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:07.203780 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27142ef4-c7f7-42fe-ae50-d8a4cfb0753a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:07.203780 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27142ef4-c7f7-42fe-ae50-d8a4cfb0753a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:07.223350 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-27142ef4-c7f7-42fe-ae50-d8a4cfb0753a None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:07.224836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-27142ef4-c7f7-42fe-ae50-d8a4cfb0753a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:07.296871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 560/1132] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:32:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:32:07.563524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec36a19-16e9-4302-bedd-357c9b938a65 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:07.621161 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:07.670176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec36a19-16e9-4302-bedd-357c9b938a65 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:07.705990 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3e270-c962-4281-addc-64e2b6f1ef9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:07.706612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3e270-c962-4281-addc-64e2b6f1ef9b None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:07.706889 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3e270-c962-4281-addc-64e2b6f1ef9b None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:07.707160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3e270-c962-4281-addc-64e2b6f1ef9b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:07.708312 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3e270-c962-4281-addc-64e2b6f1ef9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:07.709781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3e270-c962-4281-addc-64e2b6f1ef9b None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:07.711366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 573/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:32:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:07.717648 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27142ef4-c7f7-42fe-ae50-d8a4cfb0753a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:07.733892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:07.822413 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27142ef4-c7f7-42fe-ae50-d8a4cfb0753a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:07.857851 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80709f8-a457-4388-99bd-1371042b4986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:07.858503 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80709f8-a457-4388-99bd-1371042b4986 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:07.858745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80709f8-a457-4388-99bd-1371042b4986 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:07.859010 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80709f8-a457-4388-99bd-1371042b4986 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:07.859661 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80709f8-a457-4388-99bd-1371042b4986 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:07.861090 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80709f8-a457-4388-99bd-1371042b4986 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:07.888659 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 561/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 26 05:32:07 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:32:10.076858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c98021-4994-467d-868e-f3e52cf693e9 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:10.077479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c98021-4994-467d-868e-f3e52cf693e9 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:10.077479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c98021-4994-467d-868e-f3e52cf693e9 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:10.078875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 574/1135] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:32:10.084692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16466d66-73bc-4baf-85b5-19d16d1e1ff4 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:10.085174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16466d66-73bc-4baf-85b5-19d16d1e1ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:10.085295 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16466d66-73bc-4baf-85b5-19d16d1e1ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:10.104329 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-16466d66-73bc-4baf-85b5-19d16d1e1ff4 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:10.105543 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-16466d66-73bc-4baf-85b5-19d16d1e1ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:10.164509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:10.175087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 562/1136] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:32:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 05:32:10.447121 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c98021-4994-467d-868e-f3e52cf693e9 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:10.551074 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c98021-4994-467d-868e-f3e52cf693e9 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:10.571134 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:10.585925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34dccd-b00a-431e-8c38-eb047b6483ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:10.586556 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34dccd-b00a-431e-8c38-eb047b6483ae None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:10.586830 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34dccd-b00a-431e-8c38-eb047b6483ae None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:10.587096 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34dccd-b00a-431e-8c38-eb047b6483ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:10.588254 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34dccd-b00a-431e-8c38-eb047b6483ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:10.589815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34dccd-b00a-431e-8c38-eb047b6483ae None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:10.591384 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 575/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:32:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:10.597817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16466d66-73bc-4baf-85b5-19d16d1e1ff4 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:10.699192 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16466d66-73bc-4baf-85b5-19d16d1e1ff4 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:10.735157 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bfb9a-e620-4513-9ea2-da0d38411902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:10.735749 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bfb9a-e620-4513-9ea2-da0d38411902 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:10.736054 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bfb9a-e620-4513-9ea2-da0d38411902 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:10.736326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bfb9a-e620-4513-9ea2-da0d38411902 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:10.736988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22bfb9a-e620-4513-9ea2-da0d38411902 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:10.738464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22bfb9a-e620-4513-9ea2-da0d38411902 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:10.762135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 563/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:32:10 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:32:12.935066 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc717e-1915-408d-8831-f9640da09b13 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:12.935669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc717e-1915-408d-8831-f9640da09b13 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:12.935669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc717e-1915-408d-8831-f9640da09b13 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:12.936931 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 576/1139] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:32:12.942422 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc7f0a-57ab-403a-b211-77ceecfdebec None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:12.942889 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc7f0a-57ab-403a-b211-77ceecfdebec None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:12.942990 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc7f0a-57ab-403a-b211-77ceecfdebec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:12.962885 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-abfc7f0a-57ab-403a-b211-77ceecfdebec None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:12.964132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-abfc7f0a-57ab-403a-b211-77ceecfdebec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:13.022547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:13.046576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 564/1140] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:32:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:32:13.346787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc717e-1915-408d-8831-f9640da09b13 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:13.454056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc717e-1915-408d-8831-f9640da09b13 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:13.489753 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72941303-0812-4e0d-a059-df7419b356e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:13.490308 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72941303-0812-4e0d-a059-df7419b356e4 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:13.490579 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72941303-0812-4e0d-a059-df7419b356e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:13.490842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72941303-0812-4e0d-a059-df7419b356e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:13.501583 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72941303-0812-4e0d-a059-df7419b356e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:32:13.506282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-72941303-0812-4e0d-a059-df7419b356e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:32:13.506282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:32:13.506282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:32:13.506282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:32:13.506282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:32:13.506282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:32:13.506282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:32:13.506282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:32:13.506282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:32:13.506282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:32:13.506282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:32:13.506282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:32:13.506282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:32:13.506282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:32:13.506282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:32:13.506282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:32:13.507433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:32:13.507433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:32:13.507433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:32:13.507433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:32:13.507433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:32:13.507433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:32:13.507433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:32:13.507433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:32:13.507433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:32:13.507433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:32:13.507433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:32:13.507433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:32:13.507433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:32:13.507433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:32:13.507433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:32:13.507433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:32:13.508519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:32:13.508519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:32:13.508519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:32:13.508519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:32:13.508519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:32:13.508519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:32:13.508519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:32:13.508519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:32:13.508519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:32:13.508519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:32:13.508519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:32:13.508519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:32:13.508519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 577/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:32:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:32:13.513675 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc7f0a-57ab-403a-b211-77ceecfdebec None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:13.617632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc7f0a-57ab-403a-b211-77ceecfdebec None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:13.657396 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e31efa-208a-4306-9221-9d5ab4c386a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:13.658015 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e31efa-208a-4306-9221-9d5ab4c386a9 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:13.658288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e31efa-208a-4306-9221-9d5ab4c386a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:13.658555 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e31efa-208a-4306-9221-9d5ab4c386a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:13.659221 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e31efa-208a-4306-9221-9d5ab4c386a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:32:13.659519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e31efa-208a-4306-9221-9d5ab4c386a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:13.660883 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e31efa-208a-4306-9221-9d5ab4c386a9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:13.669493 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 565/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:32:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:13.675752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72941303-0812-4e0d-a059-df7419b356e4 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:13.727264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:13.779920 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72941303-0812-4e0d-a059-df7419b356e4 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:13.814691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702041eb-12b2-4437-9bc3-32c122d16c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:13.815316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702041eb-12b2-4437-9bc3-32c122d16c41 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:13.815592 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702041eb-12b2-4437-9bc3-32c122d16c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:13.815861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702041eb-12b2-4437-9bc3-32c122d16c41 None admin] PATH_INFO: `/v3/users/b4efe1aed9544f1cbdfbda92e2391f11` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:13.830932 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702041eb-12b2-4437-9bc3-32c122d16c41 None admin] RBAC: Authorizing `identity:get_user(user_id=b4efe1aed9544f1cbdfbda92e2391f11)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:13.832387 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702041eb-12b2-4437-9bc3-32c122d16c41 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:13.834052 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 578/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:32:13 2022] GET /identity/v3/users/b4efe1aed9544f1cbdfbda92e2391f11 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:13.840227 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e31efa-208a-4306-9221-9d5ab4c386a9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:13.887190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:13.947068 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e31efa-208a-4306-9221-9d5ab4c386a9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:13.984602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af434663-41ed-4ee4-bf7a-bcd2a8084d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:13.985241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af434663-41ed-4ee4-bf7a-bcd2a8084d45 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:13.985523 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af434663-41ed-4ee4-bf7a-bcd2a8084d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:13.985794 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af434663-41ed-4ee4-bf7a-bcd2a8084d45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:14.054359 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af434663-41ed-4ee4-bf7a-bcd2a8084d45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:14.055734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af434663-41ed-4ee4-bf7a-bcd2a8084d45 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:14.098323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:14.159048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 566/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:32:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:32:14.165418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702041eb-12b2-4437-9bc3-32c122d16c41 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:14.270622 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702041eb-12b2-4437-9bc3-32c122d16c41 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:14.278189 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:14.307829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f8b23-968d-4d00-a424-26878122cbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:14.308377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f8b23-968d-4d00-a424-26878122cbfc None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:14.308646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f8b23-968d-4d00-a424-26878122cbfc None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:14.308913 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f8b23-968d-4d00-a424-26878122cbfc None admin] PATH_INFO: `/v3/projects/2487f8b3b30641e0aa58f623f40a42bc` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:14.310105 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f8b23-968d-4d00-a424-26878122cbfc None admin] RBAC: Authorizing `identity:get_project(project_id=2487f8b3b30641e0aa58f623f40a42bc)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:14.311518 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f8b23-968d-4d00-a424-26878122cbfc None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:14.313168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 579/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:32:14 2022] GET /identity/v3/projects/2487f8b3b30641e0aa58f623f40a42bc => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:14.319565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af434663-41ed-4ee4-bf7a-bcd2a8084d45 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:14.423041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af434663-41ed-4ee4-bf7a-bcd2a8084d45 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:14.458612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be943129-1ac2-48de-a04f-a4d02a3fe0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:14.459221 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be943129-1ac2-48de-a04f-a4d02a3fe0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:14.459507 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be943129-1ac2-48de-a04f-a4d02a3fe0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:14.459779 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be943129-1ac2-48de-a04f-a4d02a3fe0a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:14.460505 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be943129-1ac2-48de-a04f-a4d02a3fe0a6 None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=2487f8b3b30641e0aa58f623f40a42bc, user.id=b4efe1aed9544f1cbdfbda92e2391f11) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:32:14.460816 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be943129-1ac2-48de-a04f-a4d02a3fe0a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:14.462285 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be943129-1ac2-48de-a04f-a4d02a3fe0a6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:14.468072 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 567/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:32:14 2022] GET /identity/v3/role_assignments?user.id=b4efe1aed9544f1cbdfbda92e2391f11&scope.project.id=2487f8b3b30641e0aa58f623f40a42bc&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:16.575155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f507cdd-d695-4f37-aba9-7ab6abc32e82 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:16.575729 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f507cdd-d695-4f37-aba9-7ab6abc32e82 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:16.575729 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f507cdd-d695-4f37-aba9-7ab6abc32e82 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:16.577033 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 580/1147] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:32:16.582396 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d30323-1f1f-4b8c-b5c5-b196ceb6d333 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:16.582810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d30323-1f1f-4b8c-b5c5-b196ceb6d333 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:16.582899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d30323-1f1f-4b8c-b5c5-b196ceb6d333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:16.597051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:16.603128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d1d30323-1f1f-4b8c-b5c5-b196ceb6d333 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:16.604344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d1d30323-1f1f-4b8c-b5c5-b196ceb6d333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:16.674458 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 568/1148] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:32:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:32:16.937814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f507cdd-d695-4f37-aba9-7ab6abc32e82 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:17.042716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f507cdd-d695-4f37-aba9-7ab6abc32e82 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:17.078122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aebcbc-accc-4f97-b692-fce89dc22ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:17.078741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aebcbc-accc-4f97-b692-fce89dc22ddf None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:17.079010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aebcbc-accc-4f97-b692-fce89dc22ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:17.079273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aebcbc-accc-4f97-b692-fce89dc22ddf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:17.089529 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aebcbc-accc-4f97-b692-fce89dc22ddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:32:17.094202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-47aebcbc-accc-4f97-b692-fce89dc22ddf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:32:17.094202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:32:17.094202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:32:17.094202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:32:17.094202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:32:17.094202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:32:17.094202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:32:17.094202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:32:17.094202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:32:17.094202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:32:17.094202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:32:17.094202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:32:17.094202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:32:17.094202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:32:17.094202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:32:17.094202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:32:17.095294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:32:17.095294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:32:17.095294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:32:17.095294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:32:17.095294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:32:17.095294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:32:17.095294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:32:17.095294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:32:17.095294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:32:17.095294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:32:17.095294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:32:17.095294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:32:17.095294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:32:17.095294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:32:17.095294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:32:17.095294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:32:17.096540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:32:17.096540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:32:17.096540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:32:17.096540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:32:17.096540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:32:17.096540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:32:17.096540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:32:17.096540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:32:17.096540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:32:17.096540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:32:17.096540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:32:17.096540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:32:17.096540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 581/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:32:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:32:17.102033 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d30323-1f1f-4b8c-b5c5-b196ceb6d333 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:17.203384 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d30323-1f1f-4b8c-b5c5-b196ceb6d333 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:17.238687 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1003c-f1ef-4750-b356-b87c6a5aa18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:17.239299 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1003c-f1ef-4750-b356-b87c6a5aa18e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:17.239571 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1003c-f1ef-4750-b356-b87c6a5aa18e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:17.239834 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1003c-f1ef-4750-b356-b87c6a5aa18e None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:17.240506 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1003c-f1ef-4750-b356-b87c6a5aa18e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:32:17.240803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1003c-f1ef-4750-b356-b87c6a5aa18e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:17.242195 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1003c-f1ef-4750-b356-b87c6a5aa18e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:17.244394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:17.251743 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 569/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:32:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:17.257839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aebcbc-accc-4f97-b692-fce89dc22ddf None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:17.292398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:17.361532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aebcbc-accc-4f97-b692-fce89dc22ddf None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:17.398540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6973f2-c94a-47c4-ae7d-c96b8b290e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:17.399164 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6973f2-c94a-47c4-ae7d-c96b8b290e5a None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:17.399440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6973f2-c94a-47c4-ae7d-c96b8b290e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:17.399715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6973f2-c94a-47c4-ae7d-c96b8b290e5a None admin] PATH_INFO: `/v3/users/b4efe1aed9544f1cbdfbda92e2391f11` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:17.400896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6973f2-c94a-47c4-ae7d-c96b8b290e5a None admin] RBAC: Authorizing `identity:get_user(user_id=b4efe1aed9544f1cbdfbda92e2391f11)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:17.402357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6973f2-c94a-47c4-ae7d-c96b8b290e5a None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:17.403945 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 582/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:32:17 2022] GET /identity/v3/users/b4efe1aed9544f1cbdfbda92e2391f11 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:17.410110 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1003c-f1ef-4750-b356-b87c6a5aa18e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:17.514242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1003c-f1ef-4750-b356-b87c6a5aa18e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:17.548992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570af51-0f18-4424-9f13-9421d14d78a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:17.549639 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570af51-0f18-4424-9f13-9421d14d78a4 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:17.549913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570af51-0f18-4424-9f13-9421d14d78a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:17.550174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570af51-0f18-4424-9f13-9421d14d78a4 None admin] PATH_INFO: `/v3/projects/2487f8b3b30641e0aa58f623f40a42bc` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:17.551336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570af51-0f18-4424-9f13-9421d14d78a4 None admin] RBAC: Authorizing `identity:get_project(project_id=2487f8b3b30641e0aa58f623f40a42bc)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:17.552754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570af51-0f18-4424-9f13-9421d14d78a4 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:17.554429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 570/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:32:17 2022] GET /identity/v3/projects/2487f8b3b30641e0aa58f623f40a42bc => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:17.561248 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6973f2-c94a-47c4-ae7d-c96b8b290e5a None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:17.675825 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6973f2-c94a-47c4-ae7d-c96b8b290e5a None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:17.677664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:17.717854 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69cf4b-99c5-4970-b610-55a96f4bfbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:17.718531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69cf4b-99c5-4970-b610-55a96f4bfbc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:17.718804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69cf4b-99c5-4970-b610-55a96f4bfbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:17.719069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69cf4b-99c5-4970-b610-55a96f4bfbc2 None admin] PATH_INFO: `/v3/projects/2487f8b3b30641e0aa58f623f40a42bc/users/b4efe1aed9544f1cbdfbda92e2391f11/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:17.721493 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69cf4b-99c5-4970-b610-55a96f4bfbc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2487f8b3b30641e0aa58f623f40a42bc, user_id=b4efe1aed9544f1cbdfbda92e2391f11, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:17.722934 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69cf4b-99c5-4970-b610-55a96f4bfbc2 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:17.737808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 583/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:32:17 2022] PUT /identity/v3/projects/2487f8b3b30641e0aa58f623f40a42bc/users/b4efe1aed9544f1cbdfbda92e2391f11/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:32:19.878138 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31e9bf-4a63-4aa8-8c9f-427ce7985b59 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:19.878735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31e9bf-4a63-4aa8-8c9f-427ce7985b59 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:19.878735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31e9bf-4a63-4aa8-8c9f-427ce7985b59 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:19.880031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 571/1154] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:32:19.885533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672069e2-4d91-473f-83ef-2f1a2cd94dd9 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:19.885993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672069e2-4d91-473f-83ef-2f1a2cd94dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:19.886094 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672069e2-4d91-473f-83ef-2f1a2cd94dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:19.906561 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-672069e2-4d91-473f-83ef-2f1a2cd94dd9 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:19.907750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-672069e2-4d91-473f-83ef-2f1a2cd94dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:19.976210 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 584/1155] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:32:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:32:20.245590 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31e9bf-4a63-4aa8-8c9f-427ce7985b59 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:20.314635 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:20.351042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31e9bf-4a63-4aa8-8c9f-427ce7985b59 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:20.385902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1691a26-9e58-4d12-99f4-15a2288da37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:20.386522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1691a26-9e58-4d12-99f4-15a2288da37f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:20.386791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1691a26-9e58-4d12-99f4-15a2288da37f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:20.387055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1691a26-9e58-4d12-99f4-15a2288da37f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:20.398279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1691a26-9e58-4d12-99f4-15a2288da37f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:32:20.402982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-f1691a26-9e58-4d12-99f4-15a2288da37f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:32:20.402982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:32:20.402982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:32:20.402982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:32:20.402982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:32:20.402982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:32:20.402982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:32:20.402982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:32:20.402982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:32:20.402982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:32:20.402982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:32:20.402982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:32:20.402982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:32:20.402982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:32:20.402982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:32:20.402982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:32:20.403989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:32:20.403989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:32:20.403989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:32:20.403989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:32:20.403989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:32:20.403989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:32:20.403989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:32:20.403989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:32:20.403989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:32:20.403989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:32:20.403989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:32:20.403989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:32:20.403989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:32:20.403989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:32:20.403989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:32:20.403989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:32:20.404913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:32:20.404913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:32:20.404913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:32:20.404913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:32:20.404913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:32:20.404913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:32:20.404913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:32:20.404913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:32:20.404913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:32:20.404913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:32:20.404913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:32:20.404913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:32:20.404913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 572/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:32:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:32:20.410656 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672069e2-4d91-473f-83ef-2f1a2cd94dd9 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:20.505387 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:20.515151 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672069e2-4d91-473f-83ef-2f1a2cd94dd9 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:20.550418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11252ff-e15e-4841-9cc4-9461d7149223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:20.551018 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11252ff-e15e-4841-9cc4-9461d7149223 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:20.551287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11252ff-e15e-4841-9cc4-9461d7149223 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:20.551563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11252ff-e15e-4841-9cc4-9461d7149223 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:20.552219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11252ff-e15e-4841-9cc4-9461d7149223 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:32:20.552525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11252ff-e15e-4841-9cc4-9461d7149223 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:20.553923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11252ff-e15e-4841-9cc4-9461d7149223 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:20.562033 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 585/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:32:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:20.568436 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1691a26-9e58-4d12-99f4-15a2288da37f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:20.669736 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1691a26-9e58-4d12-99f4-15a2288da37f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:20.695658 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:20.705334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73c823-061f-4e24-a656-cc9850908efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:20.705953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73c823-061f-4e24-a656-cc9850908efd None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:20.706229 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73c823-061f-4e24-a656-cc9850908efd None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:20.706496 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73c823-061f-4e24-a656-cc9850908efd None admin] PATH_INFO: `/v3/users/b4efe1aed9544f1cbdfbda92e2391f11` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:20.707611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73c823-061f-4e24-a656-cc9850908efd None admin] RBAC: Authorizing `identity:get_user(user_id=b4efe1aed9544f1cbdfbda92e2391f11)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:20.709037 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73c823-061f-4e24-a656-cc9850908efd None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:20.710653 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 573/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:32:20 2022] GET /identity/v3/users/b4efe1aed9544f1cbdfbda92e2391f11 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:20.716942 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11252ff-e15e-4841-9cc4-9461d7149223 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:20.823203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11252ff-e15e-4841-9cc4-9461d7149223 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:20.858437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfadfa3-9a41-4dea-9435-24c2bfd7a302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:20.859051 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfadfa3-9a41-4dea-9435-24c2bfd7a302 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:20.859329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfadfa3-9a41-4dea-9435-24c2bfd7a302 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:20.859612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfadfa3-9a41-4dea-9435-24c2bfd7a302 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:20.884948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:20.932925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfadfa3-9a41-4dea-9435-24c2bfd7a302 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:20.934412 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfadfa3-9a41-4dea-9435-24c2bfd7a302 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:21.034865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 586/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:32:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:32:21.041336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73c823-061f-4e24-a656-cc9850908efd None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:21.144268 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73c823-061f-4e24-a656-cc9850908efd None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:21.183335 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337046a-92cf-4f99-a4b1-c5ff36de7b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:21.183973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337046a-92cf-4f99-a4b1-c5ff36de7b64 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:21.184237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337046a-92cf-4f99-a4b1-c5ff36de7b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:21.184522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337046a-92cf-4f99-a4b1-c5ff36de7b64 None admin] PATH_INFO: `/v3/projects/2487f8b3b30641e0aa58f623f40a42bc` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:21.185716 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337046a-92cf-4f99-a4b1-c5ff36de7b64 None admin] RBAC: Authorizing `identity:get_project(project_id=2487f8b3b30641e0aa58f623f40a42bc)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:21.187180 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337046a-92cf-4f99-a4b1-c5ff36de7b64 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:21.188828 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 574/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:32:21 2022] GET /identity/v3/projects/2487f8b3b30641e0aa58f623f40a42bc => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:21.195536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfadfa3-9a41-4dea-9435-24c2bfd7a302 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:21.253747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:21.301561 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfadfa3-9a41-4dea-9435-24c2bfd7a302 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:21.338126 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773d534-0b58-40b5-bc76-bcac42cbc380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:21.338731 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773d534-0b58-40b5-bc76-bcac42cbc380 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:21.339006 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773d534-0b58-40b5-bc76-bcac42cbc380 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:21.339273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773d534-0b58-40b5-bc76-bcac42cbc380 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:21.339995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773d534-0b58-40b5-bc76-bcac42cbc380 None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=2487f8b3b30641e0aa58f623f40a42bc, user.id=b4efe1aed9544f1cbdfbda92e2391f11) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:32:21.340303 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773d534-0b58-40b5-bc76-bcac42cbc380 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:21.341774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773d534-0b58-40b5-bc76-bcac42cbc380 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:21.348071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 587/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:32:21 2022] GET /identity/v3/role_assignments?user.id=b4efe1aed9544f1cbdfbda92e2391f11&scope.project.id=2487f8b3b30641e0aa58f623f40a42bc&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:23.489354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3ed3a-4f54-476c-ba4e-d647030c5b73 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:23.489912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3ed3a-4f54-476c-ba4e-d647030c5b73 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:23.489912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3ed3a-4f54-476c-ba4e-d647030c5b73 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:23.491256 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 575/1162] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:32:23.497673 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760e487-f0ca-4d1d-8942-66e6cc141621 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:23.498119 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760e487-f0ca-4d1d-8942-66e6cc141621 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:23.498213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760e487-f0ca-4d1d-8942-66e6cc141621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:23.516786 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-3760e487-f0ca-4d1d-8942-66e6cc141621 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:23.517983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-3760e487-f0ca-4d1d-8942-66e6cc141621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:23.585163 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 588/1163] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:32:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:32:23.848599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3ed3a-4f54-476c-ba4e-d647030c5b73 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:23.913862 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:23.954920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3ed3a-4f54-476c-ba4e-d647030c5b73 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:23.990049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d60ec-c72c-4f19-825d-ae815a456ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:23.990655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d60ec-c72c-4f19-825d-ae815a456ccc None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:23.990931 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d60ec-c72c-4f19-825d-ae815a456ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:23.991202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d60ec-c72c-4f19-825d-ae815a456ccc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:24.065181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d60ec-c72c-4f19-825d-ae815a456ccc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:24.066631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d60ec-c72c-4f19-825d-ae815a456ccc None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:24.129486 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:24.170774 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 576/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:32:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:32:24.177107 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760e487-f0ca-4d1d-8942-66e6cc141621 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:24.230940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:24.282558 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760e487-f0ca-4d1d-8942-66e6cc141621 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:24.316824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba54a0-d175-4974-84d5-c94b6fe6aa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:24.317473 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba54a0-d175-4974-84d5-c94b6fe6aa76 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:24.317749 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba54a0-d175-4974-84d5-c94b6fe6aa76 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:24.318016 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba54a0-d175-4974-84d5-c94b6fe6aa76 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:24.324786 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba54a0-d175-4974-84d5-c94b6fe6aa76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:24.326263 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba54a0-d175-4974-84d5-c94b6fe6aa76 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:24.337594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-afba54a0-d175-4974-84d5-c94b6fe6aa76 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 26 05:32:24.337594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:32:24.337594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 26 05:32:24.337594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 26 05:32:24.337594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:32:24.337594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:32:24.337594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:32:24.337594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:32:24.337594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 26 05:32:24.337594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:32:24.337594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:32:24.337594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:32:24.337594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:32:24.337594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:32:24.337594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:32:24.337594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:32:24.337594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:32:24.338856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:32:24.338856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:32:24.338856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:32:24.338856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:32:24.338856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:32:24.338856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:32:24.338856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:32:24.338856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:32:24.338856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:32:24.338856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:32:24.338856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:32:24.338856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:32:24.338856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:32:24.338856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:32:24.338856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:32:24.338856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:32:24.339946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:32:24.339946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:32:24.339946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:32:24.339946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:32:24.339946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:32:24.339946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:32:24.339946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:32:24.339946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:32:24.339946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:32:24.339946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:32:24.339946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 26 05:32:24.339946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:32:24.339946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 26 05:32:24.339946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:32:24.339946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 589/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 26 05:32:24 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:32:24.345110 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d60ec-c72c-4f19-825d-ae815a456ccc None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:24.449699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d60ec-c72c-4f19-825d-ae815a456ccc None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:24.485076 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e405ab-845e-413a-b5b8-d4268a522bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:24.485711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e405ab-845e-413a-b5b8-d4268a522bff None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:24.485987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e405ab-845e-413a-b5b8-d4268a522bff None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:24.486263 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e405ab-845e-413a-b5b8-d4268a522bff None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:24.486902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e405ab-845e-413a-b5b8-d4268a522bff None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:32:24.487205 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e405ab-845e-413a-b5b8-d4268a522bff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:24.488571 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e405ab-845e-413a-b5b8-d4268a522bff None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:24.495947 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 577/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Mon Sep 26 05:32:24 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:26.640091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafcede-5101-4621-a630-f18363732074 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:26.640740 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafcede-5101-4621-a630-f18363732074 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:26.640740 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafcede-5101-4621-a630-f18363732074 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:26.642016 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 590/1167] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:32:26.647671 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832806b-8f80-4d09-8c66-c253c29c2daa None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:26.648147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832806b-8f80-4d09-8c66-c253c29c2daa None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:26.648239 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832806b-8f80-4d09-8c66-c253c29c2daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:26.666968 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f832806b-8f80-4d09-8c66-c253c29c2daa None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:26.668171 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f832806b-8f80-4d09-8c66-c253c29c2daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:26.689054 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:26.738254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 578/1168] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:32:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:32:27.002774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafcede-5101-4621-a630-f18363732074 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:27.114683 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafcede-5101-4621-a630-f18363732074 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:27.125327 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:27.152470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68347d16-a543-42ed-a188-99caa564fa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:27.153071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68347d16-a543-42ed-a188-99caa564fa48 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:27.153377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68347d16-a543-42ed-a188-99caa564fa48 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:27.153646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68347d16-a543-42ed-a188-99caa564fa48 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:27.154309 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68347d16-a543-42ed-a188-99caa564fa48 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:27.155675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68347d16-a543-42ed-a188-99caa564fa48 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:27.172174 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 591/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Mon Sep 26 05:32:27 2022] POST /identity/v3/domains => generated 267 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:32:29.345683 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4a9bc-53bc-42fa-bb5c-83a01191f5ec None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:29.346615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4a9bc-53bc-42fa-bb5c-83a01191f5ec None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:29.346615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4a9bc-53bc-42fa-bb5c-83a01191f5ec None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:29.347911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 579/1170] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:32:29.353769 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef17212-5c4a-43d5-9518-5730b26e902a None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:29.354221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef17212-5c4a-43d5-9518-5730b26e902a None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:29.354312 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef17212-5c4a-43d5-9518-5730b26e902a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:29.373716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-bef17212-5c4a-43d5-9518-5730b26e902a None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:29.374928 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-bef17212-5c4a-43d5-9518-5730b26e902a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:29.444131 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 592/1171] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:32:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:32:29.713541 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4a9bc-53bc-42fa-bb5c-83a01191f5ec None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:29.819010 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4a9bc-53bc-42fa-bb5c-83a01191f5ec None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:29.854280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f5a80-2da4-44e5-b72d-c83496b65ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:29.854902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f5a80-2da4-44e5-b72d-c83496b65ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:29.855188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f5a80-2da4-44e5-b72d-c83496b65ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:29.855452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f5a80-2da4-44e5-b72d-c83496b65ab7 None admin] PATH_INFO: `/v3/domains/ee0cf6ed5bb6425b8a0110f3298e1c71` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:29.866179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f5a80-2da4-44e5-b72d-c83496b65ab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ee0cf6ed5bb6425b8a0110f3298e1c71)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:29.867618 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f5a80-2da4-44e5-b72d-c83496b65ab7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:29.869290 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 580/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Mon Sep 26 05:32:29 2022] GET /identity/v3/domains/ee0cf6ed5bb6425b8a0110f3298e1c71 => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:29.875511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef17212-5c4a-43d5-9518-5730b26e902a None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:29.973434 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:29.977483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef17212-5c4a-43d5-9518-5730b26e902a None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:30.017859 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57018388-7cc4-490b-aaa5-5408269fb539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:30.018472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57018388-7cc4-490b-aaa5-5408269fb539 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:30.018751 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57018388-7cc4-490b-aaa5-5408269fb539 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:30.019020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57018388-7cc4-490b-aaa5-5408269fb539 None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:30.019707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57018388-7cc4-490b-aaa5-5408269fb539 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:30.021111 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57018388-7cc4-490b-aaa5-5408269fb539 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:30.049202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 593/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 26 05:32:29 2022] POST /identity/v3/projects => generated 361 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:32:32.227724 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a64f41-7c8d-48af-b9b8-1d1f704dd402 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:32.228292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a64f41-7c8d-48af-b9b8-1d1f704dd402 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:32.228292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a64f41-7c8d-48af-b9b8-1d1f704dd402 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:32.229695 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 581/1174] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:32:32.235165 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fca37-f339-40ef-b8a8-0917d73f32a3 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:32.235637 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fca37-f339-40ef-b8a8-0917d73f32a3 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:32.235740 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705fca37-f339-40ef-b8a8-0917d73f32a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:32.254413 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-705fca37-f339-40ef-b8a8-0917d73f32a3 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:32.255606 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-705fca37-f339-40ef-b8a8-0917d73f32a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:32.328527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 594/1175] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:32:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:32:32.597185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a64f41-7c8d-48af-b9b8-1d1f704dd402 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:32.616800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:32.703869 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a64f41-7c8d-48af-b9b8-1d1f704dd402 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:32.739087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba8318-924c-432d-aed1-954835057090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:32.739709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba8318-924c-432d-aed1-954835057090 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:32.739989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba8318-924c-432d-aed1-954835057090 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:32.740260 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba8318-924c-432d-aed1-954835057090 None admin] PATH_INFO: `/v3/domains/ee0cf6ed5bb6425b8a0110f3298e1c71` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:32.741458 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba8318-924c-432d-aed1-954835057090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ee0cf6ed5bb6425b8a0110f3298e1c71)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:32.742894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba8318-924c-432d-aed1-954835057090 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:32.744501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 582/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Mon Sep 26 05:32:32 2022] GET /identity/v3/domains/ee0cf6ed5bb6425b8a0110f3298e1c71 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:32.750615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705fca37-f339-40ef-b8a8-0917d73f32a3 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:32.831858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:32.853998 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705fca37-f339-40ef-b8a8-0917d73f32a3 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:32.890902 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5b0b3-63c7-4b7d-870f-678390452c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:32.891512 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5b0b3-63c7-4b7d-870f-678390452c29 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:32.891786 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5b0b3-63c7-4b7d-870f-678390452c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:32.892058 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5b0b3-63c7-4b7d-870f-678390452c29 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:32.892707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5b0b3-63c7-4b7d-870f-678390452c29 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:32.894153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5b0b3-63c7-4b7d-870f-678390452c29 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:32.916843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 595/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:32:32 2022] POST /identity/v3/users => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:32:35.089548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b87287-f8dc-45d2-b9a8-a9b56b3ccc69 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:35.090118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b87287-f8dc-45d2-b9a8-a9b56b3ccc69 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:35.090118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b87287-f8dc-45d2-b9a8-a9b56b3ccc69 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:35.091433 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 583/1178] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:32:35.097629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b5b64-8dd4-4a3a-be69-b49c769967ff None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:35.098064 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b5b64-8dd4-4a3a-be69-b49c769967ff None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:35.098064 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b5b64-8dd4-4a3a-be69-b49c769967ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:35.117690 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-dd3b5b64-8dd4-4a3a-be69-b49c769967ff None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:35.118966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-dd3b5b64-8dd4-4a3a-be69-b49c769967ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:35.191496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 596/1179] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:32:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:32:35.456532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b87287-f8dc-45d2-b9a8-a9b56b3ccc69 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:35.550679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:35.569988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b87287-f8dc-45d2-b9a8-a9b56b3ccc69 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:35.607027 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a11848-66d9-4c94-bc5b-33f4ac5843b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:35.607650 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a11848-66d9-4c94-bc5b-33f4ac5843b3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:35.607924 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a11848-66d9-4c94-bc5b-33f4ac5843b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:35.608189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a11848-66d9-4c94-bc5b-33f4ac5843b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:35.620128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a11848-66d9-4c94-bc5b-33f4ac5843b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:32:35.624845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-52a11848-66d9-4c94-bc5b-33f4ac5843b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:32:35.624845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:32:35.624845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:32:35.624845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:32:35.624845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:32:35.624845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:32:35.624845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:32:35.624845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:32:35.624845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:32:35.624845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:32:35.624845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:32:35.624845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:32:35.624845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:32:35.624845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:32:35.624845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:32:35.624845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:32:35.626029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:32:35.626029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:32:35.626029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:32:35.626029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:32:35.626029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:32:35.626029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:32:35.626029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:32:35.626029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:32:35.626029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:32:35.626029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:32:35.626029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:32:35.626029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:32:35.626029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:32:35.626029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:32:35.626029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:32:35.626029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:32:35.627366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:32:35.627366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:32:35.627366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:32:35.627366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:32:35.627366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:32:35.627366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:32:35.627366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:32:35.627366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:32:35.627366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:32:35.627366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:32:35.627366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:32:35.627366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:32:35.627366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 584/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:32:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:32:35.632266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b5b64-8dd4-4a3a-be69-b49c769967ff None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:35.701287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:35.739869 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b5b64-8dd4-4a3a-be69-b49c769967ff None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:35.776871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fed930-d2d3-43c3-8228-087bd33e45d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:35.777507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fed930-d2d3-43c3-8228-087bd33e45d5 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:35.777778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fed930-d2d3-43c3-8228-087bd33e45d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:35.778042 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fed930-d2d3-43c3-8228-087bd33e45d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:35.778698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fed930-d2d3-43c3-8228-087bd33e45d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:32:35.778991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fed930-d2d3-43c3-8228-087bd33e45d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:35.780339 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fed930-d2d3-43c3-8228-087bd33e45d5 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:35.788591 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 597/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:32:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:35.795102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a11848-66d9-4c94-bc5b-33f4ac5843b3 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:35.902080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a11848-66d9-4c94-bc5b-33f4ac5843b3 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:35.936547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b2e2b-6c84-4752-b047-5ba657066643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:35.937185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b2e2b-6c84-4752-b047-5ba657066643 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:35.937449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b2e2b-6c84-4752-b047-5ba657066643 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:35.937716 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b2e2b-6c84-4752-b047-5ba657066643 None admin] PATH_INFO: `/v3/users/1e746ba8cd1b4b3b967f8f11a309cd82` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:35.951139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b2e2b-6c84-4752-b047-5ba657066643 None admin] RBAC: Authorizing `identity:get_user(user_id=1e746ba8cd1b4b3b967f8f11a309cd82)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:35.952562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b2e2b-6c84-4752-b047-5ba657066643 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:35.954190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 585/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:32:35 2022] GET /identity/v3/users/1e746ba8cd1b4b3b967f8f11a309cd82 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:35.960425 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fed930-d2d3-43c3-8228-087bd33e45d5 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:36.067265 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fed930-d2d3-43c3-8228-087bd33e45d5 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:36.094408 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:36.104217 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850a561-2266-4fb3-8e0b-3b9d0ac2da88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:36.104829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850a561-2266-4fb3-8e0b-3b9d0ac2da88 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:36.105090 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850a561-2266-4fb3-8e0b-3b9d0ac2da88 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:36.105385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850a561-2266-4fb3-8e0b-3b9d0ac2da88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:36.176510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850a561-2266-4fb3-8e0b-3b9d0ac2da88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:36.177985 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850a561-2266-4fb3-8e0b-3b9d0ac2da88 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:36.284950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 598/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:32:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:32:36.291333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b2e2b-6c84-4752-b047-5ba657066643 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:36.294591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:36.395881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b2e2b-6c84-4752-b047-5ba657066643 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:36.430548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb0f77-4ea1-4c78-9ae4-6206ec03a414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:36.431178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb0f77-4ea1-4c78-9ae4-6206ec03a414 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:36.431458 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb0f77-4ea1-4c78-9ae4-6206ec03a414 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:36.431733 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb0f77-4ea1-4c78-9ae4-6206ec03a414 None admin] PATH_INFO: `/v3/projects/abea362319bf48ffa37001816461f6a7` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:36.432907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb0f77-4ea1-4c78-9ae4-6206ec03a414 None admin] RBAC: Authorizing `identity:get_project(project_id=abea362319bf48ffa37001816461f6a7)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:36.434368 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb0f77-4ea1-4c78-9ae4-6206ec03a414 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:36.435997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 586/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:32:36 2022] GET /identity/v3/projects/abea362319bf48ffa37001816461f6a7 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:36.442495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850a561-2266-4fb3-8e0b-3b9d0ac2da88 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:36.472772 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:36.545899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850a561-2266-4fb3-8e0b-3b9d0ac2da88 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:36.581363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a2160-1065-47be-95fe-dbdd95791650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:36.581966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a2160-1065-47be-95fe-dbdd95791650 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:36.582243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a2160-1065-47be-95fe-dbdd95791650 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:36.582514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a2160-1065-47be-95fe-dbdd95791650 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:36.583237 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a2160-1065-47be-95fe-dbdd95791650 None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=abea362319bf48ffa37001816461f6a7, user.id=1e746ba8cd1b4b3b967f8f11a309cd82) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:32:36.583542 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a2160-1065-47be-95fe-dbdd95791650 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:36.584983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a2160-1065-47be-95fe-dbdd95791650 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:36.590620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 599/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:32:36 2022] GET /identity/v3/role_assignments?user.id=1e746ba8cd1b4b3b967f8f11a309cd82&scope.project.id=abea362319bf48ffa37001816461f6a7&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:38.704447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ff482-3d34-4f83-84aa-85b2fd4aede1 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:38.705043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ff482-3d34-4f83-84aa-85b2fd4aede1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:38.705043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ff482-3d34-4f83-84aa-85b2fd4aede1 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:38.706384 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 587/1186] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:32:38.711842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde8d51-bbe5-432b-a363-dfb7a0cee588 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:38.712283 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde8d51-bbe5-432b-a363-dfb7a0cee588 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:38.712383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde8d51-bbe5-432b-a363-dfb7a0cee588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:38.736825 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-cdde8d51-bbe5-432b-a363-dfb7a0cee588 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:38.738055 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-cdde8d51-bbe5-432b-a363-dfb7a0cee588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:38.806348 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 600/1187] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:32:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:32:39.070694 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ff482-3d34-4f83-84aa-85b2fd4aede1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:39.145077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:39.179854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ff482-3d34-4f83-84aa-85b2fd4aede1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:39.215832 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af513f1f-0716-4501-af71-5b7fc16838f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:39.216466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af513f1f-0716-4501-af71-5b7fc16838f8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:39.216768 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af513f1f-0716-4501-af71-5b7fc16838f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:39.217017 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af513f1f-0716-4501-af71-5b7fc16838f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:39.228694 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af513f1f-0716-4501-af71-5b7fc16838f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:32:39.233569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-af513f1f-0716-4501-af71-5b7fc16838f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:32:39.233569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:32:39.233569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:32:39.233569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:32:39.233569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:32:39.233569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:32:39.233569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:32:39.233569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:32:39.233569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:32:39.233569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:32:39.233569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:32:39.233569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:32:39.233569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:32:39.233569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:32:39.233569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:32:39.233569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:32:39.234699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:32:39.234699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:32:39.234699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:32:39.234699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:32:39.234699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:32:39.234699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:32:39.234699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:32:39.234699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:32:39.234699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:32:39.234699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:32:39.234699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:32:39.234699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:32:39.234699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:32:39.234699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:32:39.234699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:32:39.234699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:32:39.236022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:32:39.236022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:32:39.236022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:32:39.236022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:32:39.236022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:32:39.236022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:32:39.236022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:32:39.236022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:32:39.236022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:32:39.236022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:32:39.236022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:32:39.236022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:32:39.236022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 588/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:32:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:32:39.241022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde8d51-bbe5-432b-a363-dfb7a0cee588 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:39.266388 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:39.341115 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde8d51-bbe5-432b-a363-dfb7a0cee588 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:39.375553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e698f0-cbc7-45b2-a8e9-5a3daa33b3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:39.376177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e698f0-cbc7-45b2-a8e9-5a3daa33b3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:39.376449 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e698f0-cbc7-45b2-a8e9-5a3daa33b3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:39.376715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e698f0-cbc7-45b2-a8e9-5a3daa33b3d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:39.377431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e698f0-cbc7-45b2-a8e9-5a3daa33b3d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:32:39.377726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e698f0-cbc7-45b2-a8e9-5a3daa33b3d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:39.379093 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e698f0-cbc7-45b2-a8e9-5a3daa33b3d3 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:39.388140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 601/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:32:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:39.394949 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af513f1f-0716-4501-af71-5b7fc16838f8 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:39.497101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af513f1f-0716-4501-af71-5b7fc16838f8 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:39.526125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:39.532591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4747bbb-1efb-47bb-9211-5c3b12b5f4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:39.533274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4747bbb-1efb-47bb-9211-5c3b12b5f4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:39.533555 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4747bbb-1efb-47bb-9211-5c3b12b5f4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:39.533825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4747bbb-1efb-47bb-9211-5c3b12b5f4f4 None admin] PATH_INFO: `/v3/users/1e746ba8cd1b4b3b967f8f11a309cd82` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:39.534997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4747bbb-1efb-47bb-9211-5c3b12b5f4f4 None admin] RBAC: Authorizing `identity:get_user(user_id=1e746ba8cd1b4b3b967f8f11a309cd82)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:39.536426 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4747bbb-1efb-47bb-9211-5c3b12b5f4f4 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:39.538034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 589/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:32:39 2022] GET /identity/v3/users/1e746ba8cd1b4b3b967f8f11a309cd82 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:32:39.544118 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e698f0-cbc7-45b2-a8e9-5a3daa33b3d3 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:39.630471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:39.646639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e698f0-cbc7-45b2-a8e9-5a3daa33b3d3 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:39.680855 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560e443-c723-4995-8df0-4f1cd4d438d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:39.681506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560e443-c723-4995-8df0-4f1cd4d438d4 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:39.681778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560e443-c723-4995-8df0-4f1cd4d438d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:39.682048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560e443-c723-4995-8df0-4f1cd4d438d4 None admin] PATH_INFO: `/v3/projects/abea362319bf48ffa37001816461f6a7` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:39.683256 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560e443-c723-4995-8df0-4f1cd4d438d4 None admin] RBAC: Authorizing `identity:get_project(project_id=abea362319bf48ffa37001816461f6a7)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:39.684677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560e443-c723-4995-8df0-4f1cd4d438d4 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:39.686343 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 602/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:32:39 2022] GET /identity/v3/projects/abea362319bf48ffa37001816461f6a7 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:39.692600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4747bbb-1efb-47bb-9211-5c3b12b5f4f4 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:39.797851 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4747bbb-1efb-47bb-9211-5c3b12b5f4f4 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:39.833624 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce5d3e-6b8f-43cf-a99e-0de3a0ac529e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:39.834268 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce5d3e-6b8f-43cf-a99e-0de3a0ac529e None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:39.834543 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce5d3e-6b8f-43cf-a99e-0de3a0ac529e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:39.834812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce5d3e-6b8f-43cf-a99e-0de3a0ac529e None admin] PATH_INFO: `/v3/projects/abea362319bf48ffa37001816461f6a7/users/1e746ba8cd1b4b3b967f8f11a309cd82/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:39.836740 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce5d3e-6b8f-43cf-a99e-0de3a0ac529e None admin] RBAC: Authorizing `identity:create_grant(project_id=abea362319bf48ffa37001816461f6a7, user_id=1e746ba8cd1b4b3b967f8f11a309cd82, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:39.838193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce5d3e-6b8f-43cf-a99e-0de3a0ac529e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:39.853048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 590/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:32:39 2022] PUT /identity/v3/projects/abea362319bf48ffa37001816461f6a7/users/1e746ba8cd1b4b3b967f8f11a309cd82/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:32:41.953440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7ee43-7801-4405-8e30-43e559b56780 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:41.954050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7ee43-7801-4405-8e30-43e559b56780 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:41.954050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7ee43-7801-4405-8e30-43e559b56780 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:41.955276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 603/1193] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:32:41.960835 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c3f29-f706-474c-b9ab-af0ff5ab51c1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:41.961275 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c3f29-f706-474c-b9ab-af0ff5ab51c1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:41.961421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c3f29-f706-474c-b9ab-af0ff5ab51c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:41.981439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-559c3f29-f706-474c-b9ab-af0ff5ab51c1 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:32:41.982650 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-559c3f29-f706-474c-b9ab-af0ff5ab51c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:32:42.038043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:42.053860 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 591/1194] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:32:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:32:42.324048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7ee43-7801-4405-8e30-43e559b56780 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:42.432176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7ee43-7801-4405-8e30-43e559b56780 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:42.467610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7b206-f521-458b-854c-9d0c17cb8049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:42.468240 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7b206-f521-458b-854c-9d0c17cb8049 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:42.468596 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7b206-f521-458b-854c-9d0c17cb8049 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:42.469017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7b206-f521-458b-854c-9d0c17cb8049 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:42.480778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff7b206-f521-458b-854c-9d0c17cb8049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:32:42.485693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-dff7b206-f521-458b-854c-9d0c17cb8049 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:32:42.485693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:32:42.485693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:32:42.485693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:32:42.485693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:32:42.485693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:32:42.485693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:32:42.485693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:32:42.485693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:32:42.485693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:32:42.485693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:32:42.485693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:32:42.485693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:32:42.485693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:32:42.485693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:32:42.485693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:32:42.486770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:32:42.486770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:32:42.486770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:32:42.486770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:32:42.486770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:32:42.486770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:32:42.486770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:32:42.486770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:32:42.486770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:32:42.486770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:32:42.486770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:32:42.486770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:32:42.486770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:32:42.486770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:32:42.486770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:32:42.486770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:32:42.487981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:32:42.487981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:32:42.487981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:32:42.487981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:32:42.487981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:32:42.487981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:32:42.487981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:32:42.487981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:32:42.487981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:32:42.487981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:32:42.487981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:32:42.487981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:32:42.487981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 604/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:32:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:32:42.493115 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c3f29-f706-474c-b9ab-af0ff5ab51c1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:42.591193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c3f29-f706-474c-b9ab-af0ff5ab51c1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:42.625725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890672d9-c9be-4bdd-96f7-51cfdaf7f55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:42.626318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890672d9-c9be-4bdd-96f7-51cfdaf7f55f None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:42.626591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890672d9-c9be-4bdd-96f7-51cfdaf7f55f None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:42.626856 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890672d9-c9be-4bdd-96f7-51cfdaf7f55f None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:42.627504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890672d9-c9be-4bdd-96f7-51cfdaf7f55f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:32:42.627796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890672d9-c9be-4bdd-96f7-51cfdaf7f55f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:42.629179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890672d9-c9be-4bdd-96f7-51cfdaf7f55f None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:42.637196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 592/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:32:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:42.643484 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7b206-f521-458b-854c-9d0c17cb8049 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:42.660029 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:42.750713 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7b206-f521-458b-854c-9d0c17cb8049 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:42.785567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994ef9b-1fba-4229-9e3d-f3795a650450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:42.786190 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994ef9b-1fba-4229-9e3d-f3795a650450 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:42.786569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994ef9b-1fba-4229-9e3d-f3795a650450 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:42.786842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994ef9b-1fba-4229-9e3d-f3795a650450 None admin] PATH_INFO: `/v3/users/1e746ba8cd1b4b3b967f8f11a309cd82` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:42.788014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994ef9b-1fba-4229-9e3d-f3795a650450 None admin] RBAC: Authorizing `identity:get_user(user_id=1e746ba8cd1b4b3b967f8f11a309cd82)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:42.789471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994ef9b-1fba-4229-9e3d-f3795a650450 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:42.791038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 605/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Mon Sep 26 05:32:42 2022] GET /identity/v3/users/1e746ba8cd1b4b3b967f8f11a309cd82 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:42.797166 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890672d9-c9be-4bdd-96f7-51cfdaf7f55f None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:42.848395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:42.900132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890672d9-c9be-4bdd-96f7-51cfdaf7f55f None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:42.934889 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f09b8-a7cd-4418-811d-26a98bca7bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:42.935488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f09b8-a7cd-4418-811d-26a98bca7bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:42.935754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f09b8-a7cd-4418-811d-26a98bca7bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:42.936021 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f09b8-a7cd-4418-811d-26a98bca7bf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:43.004973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f09b8-a7cd-4418-811d-26a98bca7bf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:43.006349 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f09b8-a7cd-4418-811d-26a98bca7bf9 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:43.056518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:43.108996 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 593/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:32:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:32:43.115354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994ef9b-1fba-4229-9e3d-f3795a650450 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:43.200350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:43.222002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994ef9b-1fba-4229-9e3d-f3795a650450 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:43.262807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32219f60-5c70-4984-b581-0e9c6e9e6d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:43.263355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32219f60-5c70-4984-b581-0e9c6e9e6d7a None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:43.263628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32219f60-5c70-4984-b581-0e9c6e9e6d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:43.263898 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32219f60-5c70-4984-b581-0e9c6e9e6d7a None admin] PATH_INFO: `/v3/projects/abea362319bf48ffa37001816461f6a7` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:43.265045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32219f60-5c70-4984-b581-0e9c6e9e6d7a None admin] RBAC: Authorizing `identity:get_project(project_id=abea362319bf48ffa37001816461f6a7)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:43.266509 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32219f60-5c70-4984-b581-0e9c6e9e6d7a None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:43.268167 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 606/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Mon Sep 26 05:32:43 2022] GET /identity/v3/projects/abea362319bf48ffa37001816461f6a7 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:43.274686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f09b8-a7cd-4418-811d-26a98bca7bf9 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:32:43.382271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f09b8-a7cd-4418-811d-26a98bca7bf9 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:32:43.417278 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d79b0-fee1-45e2-a83f-e3ac607793d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:32:43.417923 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d79b0-fee1-45e2-a83f-e3ac607793d5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:32:43.418193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d79b0-fee1-45e2-a83f-e3ac607793d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:32:43.418459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d79b0-fee1-45e2-a83f-e3ac607793d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:32:43.419180 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d79b0-fee1-45e2-a83f-e3ac607793d5 None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=abea362319bf48ffa37001816461f6a7, user.id=1e746ba8cd1b4b3b967f8f11a309cd82) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:32:43.419479 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d79b0-fee1-45e2-a83f-e3ac607793d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:32:43.420895 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d79b0-fee1-45e2-a83f-e3ac607793d5 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:32:43.427244 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 594/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:32:43 2022] GET /identity/v3/role_assignments?user.id=1e746ba8cd1b4b3b967f8f11a309cd82&scope.project.id=abea362319bf48ffa37001816461f6a7&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:32:53.258023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=26 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:32:53.424317 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=93 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:40.252681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab988c40-fe91-45ac-8054-a12dbb01c441 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:40.253383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab988c40-fe91-45ac-8054-a12dbb01c441 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:40.253383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab988c40-fe91-45ac-8054-a12dbb01c441 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:40.254615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 607/1201] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:33:40.270474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2db49-4afe-4949-81ab-3d50d3edd88e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:40.271065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2db49-4afe-4949-81ab-3d50d3edd88e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:40.271065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2db49-4afe-4949-81ab-3d50d3edd88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:40.293298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-e2c2db49-4afe-4949-81ab-3d50d3edd88e None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:33:40.294611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e2c2db49-4afe-4949-81ab-3d50d3edd88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:33:40.380865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 595/1202] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:33:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:33:40.679031 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab988c40-fe91-45ac-8054-a12dbb01c441 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:40.798526 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab988c40-fe91-45ac-8054-a12dbb01c441 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:40.838208 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3a252-f514-4052-be66-5b78cea90834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:40.838918 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3a252-f514-4052-be66-5b78cea90834 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:40.839230 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3a252-f514-4052-be66-5b78cea90834 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:40.839522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3a252-f514-4052-be66-5b78cea90834 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:40.851424 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3a252-f514-4052-be66-5b78cea90834 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:40.852942 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3a252-f514-4052-be66-5b78cea90834 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:40.871887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-05a3a252-f514-4052-be66-5b78cea90834 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:33:40.871887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:33:40.871887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:33:40.871887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:33:40.871887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:33:40.871887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:33:40.871887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:33:40.871887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:33:40.871887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:33:40.871887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:33:40.871887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:33:40.871887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:33:40.871887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:33:40.871887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:33:40.871887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:33:40.871887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:33:40.873062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:33:40.873062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:33:40.873062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:33:40.873062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:33:40.873062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:33:40.873062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:33:40.873062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:33:40.873062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:33:40.873062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:33:40.873062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:33:40.873062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:33:40.873062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:33:40.873062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:33:40.873062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:33:40.873062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:33:40.873062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:33:40.873062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:33:40.874667 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:33:40.874667 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:33:40.874667 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:33:40.878489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 608/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:33:40.879571 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2db49-4afe-4949-81ab-3d50d3edd88e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:41.013366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2db49-4afe-4949-81ab-3d50d3edd88e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:41.059858 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:41.066832 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47e700-403f-4b2f-94d3-7412fe20d70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:41.067498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47e700-403f-4b2f-94d3-7412fe20d70a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:41.067782 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47e700-403f-4b2f-94d3-7412fe20d70a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:41.068055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47e700-403f-4b2f-94d3-7412fe20d70a None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:41.068813 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47e700-403f-4b2f-94d3-7412fe20d70a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:41.069109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47e700-403f-4b2f-94d3-7412fe20d70a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:41.070814 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47e700-403f-4b2f-94d3-7412fe20d70a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:41.085736 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 596/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:33:41.091824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3a252-f514-4052-be66-5b78cea90834 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:41.172422 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:41.241805 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3a252-f514-4052-be66-5b78cea90834 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:41.289925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52d1d6-3c2b-44e3-98b9-581ee2585201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:41.289925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52d1d6-3c2b-44e3-98b9-581ee2585201 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:41.289925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52d1d6-3c2b-44e3-98b9-581ee2585201 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:41.289925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52d1d6-3c2b-44e3-98b9-581ee2585201 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:41.290813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52d1d6-3c2b-44e3-98b9-581ee2585201 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:41.291953 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52d1d6-3c2b-44e3-98b9-581ee2585201 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:41.329503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 609/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Mon Sep 26 05:33:41 2022] POST /identity/v3/users => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:33:43.817248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9578ee-2036-402e-83be-9342ea425347 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:43.817838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9578ee-2036-402e-83be-9342ea425347 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:43.817838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9578ee-2036-402e-83be-9342ea425347 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:43.819222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 597/1206] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:33:43.826014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74a12c-78e1-425e-b196-c0d205817cfc None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:43.826014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74a12c-78e1-425e-b196-c0d205817cfc None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:43.826014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74a12c-78e1-425e-b196-c0d205817cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:43.845414 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-9e74a12c-78e1-425e-b196-c0d205817cfc None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:33:43.847110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9e74a12c-78e1-425e-b196-c0d205817cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:33:43.941042 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 610/1207] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:33:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:33:44.236108 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9578ee-2036-402e-83be-9342ea425347 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:44.364775 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9578ee-2036-402e-83be-9342ea425347 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:44.408322 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc739061-a75c-4244-a348-1ba8d1738cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:44.409049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc739061-a75c-4244-a348-1ba8d1738cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:44.409375 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc739061-a75c-4244-a348-1ba8d1738cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:44.409687 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc739061-a75c-4244-a348-1ba8d1738cc5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:44.424054 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc739061-a75c-4244-a348-1ba8d1738cc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:33:44.429640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-cc739061-a75c-4244-a348-1ba8d1738cc5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:33:44.429640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:33:44.429640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:33:44.429640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:33:44.429640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:33:44.429640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:33:44.429640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:33:44.429640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:33:44.429640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:33:44.429640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:33:44.429640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:33:44.429640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:33:44.429640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:33:44.429640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:33:44.429640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:33:44.429640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:33:44.430732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:33:44.430732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:33:44.430732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:33:44.430732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:33:44.430732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:33:44.430732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:33:44.430732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:33:44.430732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:33:44.430732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:33:44.430732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:33:44.430732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:33:44.430732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:33:44.430732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:33:44.430732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:33:44.430732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:33:44.430732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:33:44.434232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:33:44.434232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:33:44.434232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:33:44.434232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:33:44.434232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:33:44.434232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:33:44.434232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:33:44.434232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:33:44.434232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:33:44.434232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:33:44.434232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:33:44.434232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:33:44.434232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 598/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:33:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:33:44.437304 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74a12c-78e1-425e-b196-c0d205817cfc None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:44.484026 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:44.559366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74a12c-78e1-425e-b196-c0d205817cfc None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:44.601965 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7b7b6-035b-42d1-b049-931abeab8b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:44.602671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7b7b6-035b-42d1-b049-931abeab8b7a None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:44.602957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7b7b6-035b-42d1-b049-931abeab8b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:44.603228 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7b7b6-035b-42d1-b049-931abeab8b7a None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:44.604097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb7b7b6-035b-42d1-b049-931abeab8b7a None admin] RBAC: Adding query filter params (name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:44.604470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb7b7b6-035b-42d1-b049-931abeab8b7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:44.606776 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb7b7b6-035b-42d1-b049-931abeab8b7a None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:44.616978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 611/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:33:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:33:44.624133 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc739061-a75c-4244-a348-1ba8d1738cc5 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:44.691228 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:44.755591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc739061-a75c-4244-a348-1ba8d1738cc5 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:44.796706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b709d-5339-4ef5-8e58-d761375859a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:44.797495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b709d-5339-4ef5-8e58-d761375859a8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:44.797876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b709d-5339-4ef5-8e58-d761375859a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:44.798169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b709d-5339-4ef5-8e58-d761375859a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:44.820223 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484b709d-5339-4ef5-8e58-d761375859a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:44.822064 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484b709d-5339-4ef5-8e58-d761375859a8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:44.839452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-484b709d-5339-4ef5-8e58-d761375859a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:33:44.839452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:33:44.839452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:33:44.839452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:33:44.839452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:33:44.839452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:33:44.839452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:33:44.839452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:33:44.839452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:33:44.839452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:33:44.839452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:33:44.839452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:33:44.839452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:33:44.839452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:33:44.839452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:33:44.839452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:33:44.840827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:33:44.840827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:33:44.840827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:33:44.840827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:33:44.840827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:33:44.840827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:33:44.840827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:33:44.840827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:33:44.840827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:33:44.840827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:33:44.840827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:33:44.840827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:33:44.840827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:33:44.840827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:33:44.840827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:33:44.840827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:33:44.840827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:33:44.841838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:33:44.841838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:33:44.841838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:33:44.841838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 599/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:33:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:33:44.847318 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7b7b6-035b-42d1-b049-931abeab8b7a None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:44.967324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:44.975246 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7b7b6-035b-42d1-b049-931abeab8b7a None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:45.016696 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7530390-0e28-4c33-b23e-a0946326d17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:45.017251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7530390-0e28-4c33-b23e-a0946326d17c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:45.017531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7530390-0e28-4c33-b23e-a0946326d17c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:45.017802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7530390-0e28-4c33-b23e-a0946326d17c None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:45.018502 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7530390-0e28-4c33-b23e-a0946326d17c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:45.018802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7530390-0e28-4c33-b23e-a0946326d17c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:45.020168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7530390-0e28-4c33-b23e-a0946326d17c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:45.032920 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 612/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:33:45.038895 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b709d-5339-4ef5-8e58-d761375859a8 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:45.154761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b709d-5339-4ef5-8e58-d761375859a8 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:45.156699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:45.195868 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba978ca-2bcf-4deb-aba0-4493d036bd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:45.196506 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba978ca-2bcf-4deb-aba0-4493d036bd71 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:45.196788 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba978ca-2bcf-4deb-aba0-4493d036bd71 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:45.197066 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba978ca-2bcf-4deb-aba0-4493d036bd71 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:45.207498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba978ca-2bcf-4deb-aba0-4493d036bd71 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:45.209108 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba978ca-2bcf-4deb-aba0-4493d036bd71 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:45.225242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-dba978ca-2bcf-4deb-aba0-4493d036bd71 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:33:45.225242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:33:45.225242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:33:45.225242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:33:45.225242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:33:45.225242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:33:45.225242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:33:45.225242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:33:45.225242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:33:45.225242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:33:45.225242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:33:45.225242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:33:45.225242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:33:45.225242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:33:45.225242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:33:45.225242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:33:45.232057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:33:45.233085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:33:45.233085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:33:45.233085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:33:45.233085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:33:45.233085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:33:45.233085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:33:45.233085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:33:45.233085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:33:45.233085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:33:45.233085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:33:45.233085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 600/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:33:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:33:45.233778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7530390-0e28-4c33-b23e-a0946326d17c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:45.368172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7530390-0e28-4c33-b23e-a0946326d17c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:45.413605 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb926a9-0668-4fda-9051-79b38b628d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:45.414234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb926a9-0668-4fda-9051-79b38b628d82 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:45.414513 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb926a9-0668-4fda-9051-79b38b628d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:45.414786 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb926a9-0668-4fda-9051-79b38b628d82 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:45.415538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb926a9-0668-4fda-9051-79b38b628d82 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:45.415845 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb926a9-0668-4fda-9051-79b38b628d82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:45.417404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb926a9-0668-4fda-9051-79b38b628d82 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:45.439465 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 613/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Sep 26 05:33:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:33:45.445698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba978ca-2bcf-4deb-aba0-4493d036bd71 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:45.572685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba978ca-2bcf-4deb-aba0-4493d036bd71 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:45.623226 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361579f6-7d33-44e0-93f2-e99360ae1213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:45.623741 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361579f6-7d33-44e0-93f2-e99360ae1213 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:45.624023 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361579f6-7d33-44e0-93f2-e99360ae1213 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:45.624285 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361579f6-7d33-44e0-93f2-e99360ae1213 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:45.660444 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:45.708289 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361579f6-7d33-44e0-93f2-e99360ae1213 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:45.710292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361579f6-7d33-44e0-93f2-e99360ae1213 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:45.853496 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 601/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:33:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:33:45.860775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb926a9-0668-4fda-9051-79b38b628d82 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:45.883352 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:45.991483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb926a9-0668-4fda-9051-79b38b628d82 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:46.036289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05067da9-a9a8-49ef-beec-a54de610b390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:46.036950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05067da9-a9a8-49ef-beec-a54de610b390 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:46.037274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05067da9-a9a8-49ef-beec-a54de610b390 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:46.037543 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05067da9-a9a8-49ef-beec-a54de610b390 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:46.049435 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05067da9-a9a8-49ef-beec-a54de610b390 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:46.050987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05067da9-a9a8-49ef-beec-a54de610b390 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:46.067365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-05067da9-a9a8-49ef-beec-a54de610b390 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:33:46.067365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:33:46.067365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:33:46.067365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:33:46.067365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:33:46.067365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:33:46.067365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:33:46.067365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:33:46.067365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:33:46.067365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:33:46.067365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:33:46.067365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:33:46.067365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:33:46.067365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:33:46.067365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:33:46.067365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:33:46.068423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:33:46.068423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:33:46.068423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:33:46.068423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:33:46.068423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:33:46.068423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:33:46.068423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:33:46.068423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:33:46.068423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:33:46.068423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:33:46.068423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:33:46.068423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:33:46.068423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:33:46.068423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:33:46.068423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:33:46.068423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:33:46.068423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:33:46.069441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:33:46.069441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:33:46.069441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:33:46.069441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 614/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:33:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:33:46.074592 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361579f6-7d33-44e0-93f2-e99360ae1213 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:46.145201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:46.188163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361579f6-7d33-44e0-93f2-e99360ae1213 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:46.234051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40705502-2522-4429-8718-1df8591a00f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:46.234742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40705502-2522-4429-8718-1df8591a00f6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:46.235024 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40705502-2522-4429-8718-1df8591a00f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:46.235370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40705502-2522-4429-8718-1df8591a00f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:46.236112 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40705502-2522-4429-8718-1df8591a00f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:46.236418 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40705502-2522-4429-8718-1df8591a00f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:46.238002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40705502-2522-4429-8718-1df8591a00f6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:46.251309 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 602/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:33:46.257316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05067da9-a9a8-49ef-beec-a54de610b390 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:46.335146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:46.368022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05067da9-a9a8-49ef-beec-a54de610b390 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:46.403736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd5743-a0bb-4f4e-a4f9-4bf6bbdad2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:46.404405 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd5743-a0bb-4f4e-a4f9-4bf6bbdad2a5 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:46.404763 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd5743-a0bb-4f4e-a4f9-4bf6bbdad2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:46.405245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd5743-a0bb-4f4e-a4f9-4bf6bbdad2a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:46.415032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd5743-a0bb-4f4e-a4f9-4bf6bbdad2a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:46.417086 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd5743-a0bb-4f4e-a4f9-4bf6bbdad2a5 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:46.432332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-48cd5743-a0bb-4f4e-a4f9-4bf6bbdad2a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:33:46.432332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:33:46.432332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:33:46.432332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:33:46.432332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:33:46.432332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:33:46.432332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:33:46.432332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:33:46.432332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:33:46.432332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:33:46.432332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:33:46.432332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:33:46.432332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:33:46.432332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:33:46.432332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:33:46.434380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:33:46.434380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:33:46.434380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:33:46.434380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:33:46.434380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:33:46.434380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:33:46.434380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:33:46.434380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:33:46.434380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:33:46.434380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:33:46.434380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:33:46.434380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:33:46.434380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:33:46.434380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:33:46.434380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:33:46.434380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:33:46.434380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:33:46.435515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:33:46.435515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:33:46.435515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:33:46.435515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:33:46.435515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:33:46.435515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:33:46.435515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:33:46.435515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:33:46.435515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 615/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:33:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:33:46.440547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40705502-2522-4429-8718-1df8591a00f6 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:46.552953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40705502-2522-4429-8718-1df8591a00f6 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:46.579263 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:46.592053 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde6f69-3905-4244-a2ac-22023b88df97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:46.592658 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde6f69-3905-4244-a2ac-22023b88df97 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:46.592932 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde6f69-3905-4244-a2ac-22023b88df97 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:46.593224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde6f69-3905-4244-a2ac-22023b88df97 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:46.593908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde6f69-3905-4244-a2ac-22023b88df97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:46.594204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde6f69-3905-4244-a2ac-22023b88df97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:46.595781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde6f69-3905-4244-a2ac-22023b88df97 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:46.611743 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 603/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:33:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:33:46.618517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd5743-a0bb-4f4e-a4f9-4bf6bbdad2a5 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:46.727964 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd5743-a0bb-4f4e-a4f9-4bf6bbdad2a5 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:46.761345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:46.768803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305fce5-8729-4578-a95b-7d2e921c2554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:46.769453 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305fce5-8729-4578-a95b-7d2e921c2554 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:46.769734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305fce5-8729-4578-a95b-7d2e921c2554 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:46.770007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305fce5-8729-4578-a95b-7d2e921c2554 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:46.770730 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305fce5-8729-4578-a95b-7d2e921c2554 None admin] RBAC: Adding query filter params (role.id=881210e567974dc0aa3043393c00a838, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=67fcd65eeb994b2381b83480e030f2ae) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:46.771042 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305fce5-8729-4578-a95b-7d2e921c2554 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:46.772586 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305fce5-8729-4578-a95b-7d2e921c2554 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:46.779044 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 616/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:33:46 2022] GET /identity/v3/role_assignments?user.id=67fcd65eeb994b2381b83480e030f2ae&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=881210e567974dc0aa3043393c00a838 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:33:49.074208 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f50272-661c-45b1-8dcf-bd256055a567 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:49.074996 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f50272-661c-45b1-8dcf-bd256055a567 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:49.074996 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f50272-661c-45b1-8dcf-bd256055a567 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:49.076597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 604/1220] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:33:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:33:49.082580 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fa11c-70be-4ce8-b4cd-dcc8a02f6aa6 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:49.083119 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fa11c-70be-4ce8-b4cd-dcc8a02f6aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:49.083119 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fa11c-70be-4ce8-b4cd-dcc8a02f6aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:49.107873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-2e1fa11c-70be-4ce8-b4cd-dcc8a02f6aa6 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:33:49.109542 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2e1fa11c-70be-4ce8-b4cd-dcc8a02f6aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:33:49.202618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 617/1221] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:33:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:33:49.511490 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f50272-661c-45b1-8dcf-bd256055a567 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:49.655575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f50272-661c-45b1-8dcf-bd256055a567 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:49.703182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806f8f0-a00b-41ec-a162-c5ef7466c022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:49.704258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806f8f0-a00b-41ec-a162-c5ef7466c022 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:49.704610 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806f8f0-a00b-41ec-a162-c5ef7466c022 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:49.705106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806f8f0-a00b-41ec-a162-c5ef7466c022 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:49.721770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806f8f0-a00b-41ec-a162-c5ef7466c022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:33:49.729067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-b806f8f0-a00b-41ec-a162-c5ef7466c022 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:33:49.729067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:33:49.729067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:33:49.729067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:33:49.729067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:33:49.729067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:33:49.729067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:33:49.729067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:33:49.729067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:33:49.729067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:33:49.729067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:33:49.729067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:33:49.729067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:33:49.729067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:33:49.729067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:33:49.729067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:33:49.731201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:33:49.731201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:33:49.731201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:33:49.731201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:33:49.731201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:33:49.731201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:33:49.731201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:33:49.731201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:33:49.731201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:33:49.731201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:33:49.731201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:33:49.731201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:33:49.731201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:33:49.731201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:33:49.731201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:33:49.731201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:33:49.732572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:33:49.732572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:33:49.732572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:33:49.732572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:33:49.732572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:33:49.732572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:33:49.732572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:33:49.732572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:33:49.732572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:33:49.732572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:33:49.732572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:33:49.732572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:33:49.732572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 605/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:33:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:33:49.740178 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fa11c-70be-4ce8-b4cd-dcc8a02f6aa6 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:49.874796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fa11c-70be-4ce8-b4cd-dcc8a02f6aa6 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:49.907522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:49.926811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68d023-7a6f-4963-bedd-cd5b625d2c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:49.927533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68d023-7a6f-4963-bedd-cd5b625d2c8a None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:49.927814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68d023-7a6f-4963-bedd-cd5b625d2c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:49.928089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68d023-7a6f-4963-bedd-cd5b625d2c8a None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:49.928802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68d023-7a6f-4963-bedd-cd5b625d2c8a None admin] RBAC: Adding query filter params (name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:49.929108 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68d023-7a6f-4963-bedd-cd5b625d2c8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:49.931155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68d023-7a6f-4963-bedd-cd5b625d2c8a None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:49.943585 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 618/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:33:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:33:49.951677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806f8f0-a00b-41ec-a162-c5ef7466c022 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:50.027690 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:50.130924 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806f8f0-a00b-41ec-a162-c5ef7466c022 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:50.183455 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31913ce-1fc5-44ff-b8dd-c88c9323f523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:50.184218 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31913ce-1fc5-44ff-b8dd-c88c9323f523 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:50.184545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31913ce-1fc5-44ff-b8dd-c88c9323f523 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:50.184826 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31913ce-1fc5-44ff-b8dd-c88c9323f523 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:50.199335 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31913ce-1fc5-44ff-b8dd-c88c9323f523 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:50.201296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31913ce-1fc5-44ff-b8dd-c88c9323f523 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:50.224459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-a31913ce-1fc5-44ff-b8dd-c88c9323f523 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:33:50.224459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:33:50.224459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:33:50.224459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:33:50.224459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:33:50.224459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:33:50.224459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:33:50.224459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:33:50.224459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:33:50.224459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:33:50.224459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:33:50.224459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:33:50.224459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:33:50.224459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:33:50.224459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:33:50.224459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:33:50.226729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:33:50.226729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:33:50.226729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:33:50.226729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:33:50.226729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:33:50.226729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:33:50.226729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:33:50.226729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:33:50.226729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:33:50.226729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:33:50.226729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:33:50.226729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:33:50.226729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:33:50.226729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:33:50.226729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:33:50.226729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:33:50.226729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:33:50.228731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:33:50.228731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:33:50.228731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:33:50.228731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 606/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:33:50.232726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68d023-7a6f-4963-bedd-cd5b625d2c8a None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:50.381514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68d023-7a6f-4963-bedd-cd5b625d2c8a None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:50.428081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa3d93-f5c8-4584-9a51-42ccc465a485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:50.428773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa3d93-f5c8-4584-9a51-42ccc465a485 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:50.429048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa3d93-f5c8-4584-9a51-42ccc465a485 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:50.429348 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa3d93-f5c8-4584-9a51-42ccc465a485 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:50.430074 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa3d93-f5c8-4584-9a51-42ccc465a485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:50.430518 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa3d93-f5c8-4584-9a51-42ccc465a485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:50.432366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa3d93-f5c8-4584-9a51-42ccc465a485 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:50.447509 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 619/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:33:50.455213 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31913ce-1fc5-44ff-b8dd-c88c9323f523 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:50.602992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:50.613165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31913ce-1fc5-44ff-b8dd-c88c9323f523 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:50.653504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2224a-e5fe-4de0-8e64-a2284335020d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:50.654042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2224a-e5fe-4de0-8e64-a2284335020d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:50.654315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2224a-e5fe-4de0-8e64-a2284335020d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:50.654582 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2224a-e5fe-4de0-8e64-a2284335020d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:50.663280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2224a-e5fe-4de0-8e64-a2284335020d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:50.664738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2224a-e5fe-4de0-8e64-a2284335020d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:50.681531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-79b2224a-e5fe-4de0-8e64-a2284335020d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:33:50.681531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:33:50.681531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:33:50.681531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:33:50.681531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:33:50.681531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:33:50.681531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:33:50.681531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:33:50.681531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:33:50.681531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:33:50.681531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:33:50.681531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:33:50.681531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:33:50.681531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:33:50.681531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:33:50.681531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:33:50.682846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:33:50.683943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:33:50.683943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:33:50.683943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:33:50.683943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:33:50.683943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:33:50.683943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:33:50.683943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:33:50.683943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:33:50.683943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:33:50.683943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:33:50.683943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 607/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:33:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:33:50.690531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa3d93-f5c8-4584-9a51-42ccc465a485 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:50.756656 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:50.847597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa3d93-f5c8-4584-9a51-42ccc465a485 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:50.904060 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c4094-0cc0-4c20-b614-08e1c91d5aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:50.904870 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c4094-0cc0-4c20-b614-08e1c91d5aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:50.905340 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c4094-0cc0-4c20-b614-08e1c91d5aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:50.905846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c4094-0cc0-4c20-b614-08e1c91d5aa4 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:50.906883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c4094-0cc0-4c20-b614-08e1c91d5aa4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:50.907531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c4094-0cc0-4c20-b614-08e1c91d5aa4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:50.909599 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c4094-0cc0-4c20-b614-08e1c91d5aa4 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:50.935505 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 620/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Sep 26 05:33:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:33:50.942192 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2224a-e5fe-4de0-8e64-a2284335020d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:51.090035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2224a-e5fe-4de0-8e64-a2284335020d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:51.152972 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbfaa2-42ed-4e79-bd9e-2edff6562d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:51.154188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbfaa2-42ed-4e79-bd9e-2edff6562d8d None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:51.154767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbfaa2-42ed-4e79-bd9e-2edff6562d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:51.155149 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbfaa2-42ed-4e79-bd9e-2edff6562d8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:51.175652 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbfaa2-42ed-4e79-bd9e-2edff6562d8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:51.178463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbfaa2-42ed-4e79-bd9e-2edff6562d8d None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:51.206982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-17bbfaa2-42ed-4e79-bd9e-2edff6562d8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:33:51.206982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:33:51.206982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:33:51.206982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:33:51.206982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:33:51.206982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:33:51.206982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:33:51.206982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:33:51.206982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:33:51.206982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:33:51.206982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:33:51.206982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:33:51.206982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:33:51.206982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:33:51.206982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:33:51.206982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:33:51.209121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:33:51.209121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:33:51.209121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:33:51.209121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:33:51.209121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:33:51.209121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:33:51.209121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:33:51.209121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:33:51.209121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:33:51.209121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:33:51.209121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:33:51.209121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:33:51.209121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:33:51.209121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:33:51.209121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:33:51.209121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:33:51.209121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:33:51.210867 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:33:51.210867 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:33:51.210867 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:33:51.210867 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 608/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:33:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:33:51.215835 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c4094-0cc0-4c20-b614-08e1c91d5aa4 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:51.389062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c4094-0cc0-4c20-b614-08e1c91d5aa4 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:51.393554 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:51.440091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785aa75c-aa10-443d-bf0a-dce3dafcc419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:51.440933 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aa75c-aa10-443d-bf0a-dce3dafcc419 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:51.441313 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aa75c-aa10-443d-bf0a-dce3dafcc419 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:51.441705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aa75c-aa10-443d-bf0a-dce3dafcc419 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:51.442503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785aa75c-aa10-443d-bf0a-dce3dafcc419 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:51.442823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785aa75c-aa10-443d-bf0a-dce3dafcc419 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:51.444291 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785aa75c-aa10-443d-bf0a-dce3dafcc419 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:51.458247 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 621/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:33:51.465722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbfaa2-42ed-4e79-bd9e-2edff6562d8d None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:51.483986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:51.598722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbfaa2-42ed-4e79-bd9e-2edff6562d8d None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:51.647607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b31a1-38e1-451f-a6a4-9889ca3820bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:51.648918 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b31a1-38e1-451f-a6a4-9889ca3820bf None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:51.651410 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b31a1-38e1-451f-a6a4-9889ca3820bf None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:51.651706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b31a1-38e1-451f-a6a4-9889ca3820bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:51.660321 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b31a1-38e1-451f-a6a4-9889ca3820bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:51.662147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b31a1-38e1-451f-a6a4-9889ca3820bf None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:51.682809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-556b31a1-38e1-451f-a6a4-9889ca3820bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:33:51.682809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:33:51.682809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:33:51.682809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:33:51.682809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:33:51.682809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:33:51.682809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:33:51.682809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:33:51.682809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:33:51.682809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:33:51.682809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:33:51.682809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:33:51.682809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:33:51.682809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:33:51.682809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:33:51.683986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:33:51.683986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:33:51.683986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:33:51.683986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:33:51.683986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:33:51.683986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:33:51.683986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:33:51.683986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:33:51.683986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:33:51.683986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:33:51.683986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:33:51.683986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:33:51.683986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:33:51.683986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:33:51.683986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:33:51.683986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:33:51.683986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:33:51.685178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:33:51.685178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:33:51.685178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:33:51.685178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:33:51.685178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:33:51.685178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:33:51.685178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:33:51.685178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:33:51.685178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 609/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:33:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:33:51.696213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785aa75c-aa10-443d-bf0a-dce3dafcc419 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:51.876225 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785aa75c-aa10-443d-bf0a-dce3dafcc419 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:51.940087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021771d-01c3-40f4-a924-fa9d9a759035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:51.941387 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021771d-01c3-40f4-a924-fa9d9a759035 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:51.941832 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021771d-01c3-40f4-a924-fa9d9a759035 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:51.942223 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021771d-01c3-40f4-a924-fa9d9a759035 None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:51.943132 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021771d-01c3-40f4-a924-fa9d9a759035 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:51.943523 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021771d-01c3-40f4-a924-fa9d9a759035 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:51.945166 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021771d-01c3-40f4-a924-fa9d9a759035 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:51.963064 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 622/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:33:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:33:51.972183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b31a1-38e1-451f-a6a4-9889ca3820bf None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:52.075642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:52.132892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b31a1-38e1-451f-a6a4-9889ca3820bf None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:52.199751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bf23b-bcdc-4068-b947-31a42b5b4c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:52.200923 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bf23b-bcdc-4068-b947-31a42b5b4c4b None admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:52.201515 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bf23b-bcdc-4068-b947-31a42b5b4c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:52.202043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bf23b-bcdc-4068-b947-31a42b5b4c4b None admin] PATH_INFO: `/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/67fcd65eeb994b2381b83480e030f2ae/roles/881210e567974dc0aa3043393c00a838` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:52.235036 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977bf23b-bcdc-4068-b947-31a42b5b4c4b None admin] RBAC: Authorizing `identity:create_grant(project_id=5e44d601292b47b7b66d6f3efa80b450, user_id=67fcd65eeb994b2381b83480e030f2ae, role_id=881210e567974dc0aa3043393c00a838)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:52.237429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977bf23b-bcdc-4068-b947-31a42b5b4c4b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:52.261504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 610/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:33:51 2022] PUT /identity/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/67fcd65eeb994b2381b83480e030f2ae/roles/881210e567974dc0aa3043393c00a838 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:33:54.745939 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407f088-b97e-4510-adf5-df4c2a01e911 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:54.746906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407f088-b97e-4510-adf5-df4c2a01e911 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:54.746906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407f088-b97e-4510-adf5-df4c2a01e911 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:54.747846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 623/1233] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:33:54.756772 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15355e28-147e-4a54-b139-dd267426957d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:54.758147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15355e28-147e-4a54-b139-dd267426957d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:54.758147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15355e28-147e-4a54-b139-dd267426957d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:54.789052 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-15355e28-147e-4a54-b139-dd267426957d None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:33:54.790478 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-15355e28-147e-4a54-b139-dd267426957d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:33:54.877115 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 611/1234] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:33:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:33:55.214816 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407f088-b97e-4510-adf5-df4c2a01e911 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:55.261644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:55.382862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407f088-b97e-4510-adf5-df4c2a01e911 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:55.443654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b336f-2ae7-4942-95e9-82898ec849c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:55.444298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b336f-2ae7-4942-95e9-82898ec849c0 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:55.444580 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b336f-2ae7-4942-95e9-82898ec849c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:55.444828 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b336f-2ae7-4942-95e9-82898ec849c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:55.460277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b336f-2ae7-4942-95e9-82898ec849c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:33:55.466503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-d40b336f-2ae7-4942-95e9-82898ec849c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:33:55.466503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:33:55.466503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:33:55.466503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:33:55.466503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:33:55.466503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:33:55.466503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:33:55.466503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:33:55.466503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:33:55.466503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:33:55.466503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:33:55.466503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:33:55.466503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:33:55.466503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:33:55.466503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:33:55.466503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:33:55.468182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:33:55.468182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:33:55.468182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:33:55.468182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:33:55.468182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:33:55.468182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:33:55.468182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:33:55.468182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:33:55.468182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:33:55.468182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:33:55.468182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:33:55.468182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:33:55.468182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:33:55.468182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:33:55.468182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:33:55.468182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:33:55.469286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:33:55.469286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:33:55.469286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:33:55.469286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:33:55.469286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:33:55.469286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:33:55.469286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:33:55.469286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:33:55.469286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:33:55.469286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:33:55.469286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 05:33:55.469286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:33:55.469286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 624/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Mon Sep 26 05:33:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:33:55.476010 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15355e28-147e-4a54-b139-dd267426957d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:55.567871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:55.644876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15355e28-147e-4a54-b139-dd267426957d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:55.692450 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c6bf0-964e-4836-aedd-aa5d31e23ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:55.693338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c6bf0-964e-4836-aedd-aa5d31e23ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:55.693587 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c6bf0-964e-4836-aedd-aa5d31e23ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:55.693832 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c6bf0-964e-4836-aedd-aa5d31e23ae7 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:55.694521 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c6bf0-964e-4836-aedd-aa5d31e23ae7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:55.694823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c6bf0-964e-4836-aedd-aa5d31e23ae7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:55.696401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c6bf0-964e-4836-aedd-aa5d31e23ae7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:55.707908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 612/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Mon Sep 26 05:33:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:33:55.715034 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b336f-2ae7-4942-95e9-82898ec849c0 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:55.841664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:55.862359 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b336f-2ae7-4942-95e9-82898ec849c0 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:55.910652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7fd63-dcd7-4327-b2e9-29a48c980b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:55.911541 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7fd63-dcd7-4327-b2e9-29a48c980b44 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:55.911708 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7fd63-dcd7-4327-b2e9-29a48c980b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:55.911933 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7fd63-dcd7-4327-b2e9-29a48c980b44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:55.928668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7fd63-dcd7-4327-b2e9-29a48c980b44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:55.930825 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7fd63-dcd7-4327-b2e9-29a48c980b44 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:55.949510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-46e7fd63-dcd7-4327-b2e9-29a48c980b44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:33:55.949510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:33:55.949510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:33:55.949510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:33:55.949510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:33:55.949510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:33:55.949510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:33:55.949510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:33:55.949510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:33:55.949510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:33:55.949510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:33:55.949510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:33:55.949510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:33:55.949510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:33:55.949510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:33:55.949510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:33:55.951467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:33:55.951467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:33:55.951467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:33:55.951467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:33:55.951467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:33:55.951467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:33:55.951467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:33:55.951467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:33:55.951467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:33:55.951467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:33:55.951467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:33:55.951467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:33:55.951467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:33:55.951467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:33:55.951467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:33:55.951467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:33:55.951467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:33:55.952748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:33:55.952748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:33:55.952748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:33:55.952748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 625/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:33:55.956977 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c6bf0-964e-4836-aedd-aa5d31e23ae7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:56.109993 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c6bf0-964e-4836-aedd-aa5d31e23ae7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:56.124332 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:56.162092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc9fed-8257-49d0-bc2f-bac4f32b4395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:56.162805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc9fed-8257-49d0-bc2f-bac4f32b4395 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:56.163252 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc9fed-8257-49d0-bc2f-bac4f32b4395 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:56.163532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc9fed-8257-49d0-bc2f-bac4f32b4395 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:56.164438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc9fed-8257-49d0-bc2f-bac4f32b4395 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:56.164963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc9fed-8257-49d0-bc2f-bac4f32b4395 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:56.166907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc9fed-8257-49d0-bc2f-bac4f32b4395 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:56.184438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 613/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:33:56.193782 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7fd63-dcd7-4327-b2e9-29a48c980b44 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:56.340622 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7fd63-dcd7-4327-b2e9-29a48c980b44 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:56.391977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d6d34-b9cf-4975-924e-5a54dd2e926f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:56.392867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d6d34-b9cf-4975-924e-5a54dd2e926f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:56.393249 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d6d34-b9cf-4975-924e-5a54dd2e926f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:56.393557 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d6d34-b9cf-4975-924e-5a54dd2e926f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:56.397628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:56.405610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d6d34-b9cf-4975-924e-5a54dd2e926f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:56.407620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d6d34-b9cf-4975-924e-5a54dd2e926f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:56.423972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-f63d6d34-b9cf-4975-924e-5a54dd2e926f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:33:56.423972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:33:56.423972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:33:56.423972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:33:56.423972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:33:56.423972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:33:56.423972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:33:56.423972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:33:56.423972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:33:56.423972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:33:56.423972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:33:56.423972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:33:56.423972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:33:56.423972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:33:56.423972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:33:56.423972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:33:56.426088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:33:56.428091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:33:56.428091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:33:56.428091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:33:56.428091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:33:56.428091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:33:56.428091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:33:56.428091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:33:56.428091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:33:56.428091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:33:56.428091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:33:56.428091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 626/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:33:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:33:56.432942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc9fed-8257-49d0-bc2f-bac4f32b4395 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:56.583689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc9fed-8257-49d0-bc2f-bac4f32b4395 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:56.635915 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb08f0a-b250-4a64-8570-4bb7395ae50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:56.636702 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb08f0a-b250-4a64-8570-4bb7395ae50e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:56.637248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb08f0a-b250-4a64-8570-4bb7395ae50e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:56.637681 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb08f0a-b250-4a64-8570-4bb7395ae50e None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:56.638887 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb08f0a-b250-4a64-8570-4bb7395ae50e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:56.639402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb08f0a-b250-4a64-8570-4bb7395ae50e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:56.641643 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb08f0a-b250-4a64-8570-4bb7395ae50e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:56.670587 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 614/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Sep 26 05:33:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:33:56.678032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d6d34-b9cf-4975-924e-5a54dd2e926f None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:56.826054 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d6d34-b9cf-4975-924e-5a54dd2e926f None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:56.881446 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300f185-643e-4036-b29b-74433e5c99e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:56.882553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300f185-643e-4036-b29b-74433e5c99e7 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:56.884479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300f185-643e-4036-b29b-74433e5c99e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:56.884479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300f185-643e-4036-b29b-74433e5c99e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:56.991293 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:57.004172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300f185-643e-4036-b29b-74433e5c99e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:57.006190 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300f185-643e-4036-b29b-74433e5c99e7 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:57.174713 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 627/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:33:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:33:57.181472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb08f0a-b250-4a64-8570-4bb7395ae50e None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:57.229101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:57.323276 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb08f0a-b250-4a64-8570-4bb7395ae50e None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:57.377892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f14b89-94fc-49e2-aa6c-27cc63612f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:57.379196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f14b89-94fc-49e2-aa6c-27cc63612f4a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:57.379688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f14b89-94fc-49e2-aa6c-27cc63612f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:57.380199 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f14b89-94fc-49e2-aa6c-27cc63612f4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:57.401904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f14b89-94fc-49e2-aa6c-27cc63612f4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:57.403904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f14b89-94fc-49e2-aa6c-27cc63612f4a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:57.423247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-d8f14b89-94fc-49e2-aa6c-27cc63612f4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:33:57.423247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:33:57.423247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:33:57.423247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:33:57.423247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:33:57.423247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:33:57.423247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:33:57.423247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:33:57.423247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:33:57.423247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:33:57.423247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:33:57.423247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:33:57.423247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:33:57.423247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:33:57.423247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:33:57.423247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:33:57.425439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:33:57.425439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:33:57.425439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:33:57.425439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:33:57.425439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:33:57.425439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:33:57.425439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:33:57.425439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:33:57.425439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:33:57.425439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:33:57.425439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:33:57.425439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:33:57.425439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:33:57.425439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:33:57.425439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:33:57.425439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:33:57.425439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:33:57.427267 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:33:57.427267 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:33:57.427267 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:33:57.427267 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 615/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:33:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:33:57.434310 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300f185-643e-4036-b29b-74433e5c99e7 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:57.618908 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:57.630133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300f185-643e-4036-b29b-74433e5c99e7 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:57.688536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44616bf5-c227-48da-b45e-a00dc78c0688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:57.689845 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44616bf5-c227-48da-b45e-a00dc78c0688 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:57.690201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44616bf5-c227-48da-b45e-a00dc78c0688 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:57.690626 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44616bf5-c227-48da-b45e-a00dc78c0688 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:57.691520 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44616bf5-c227-48da-b45e-a00dc78c0688 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:57.691957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44616bf5-c227-48da-b45e-a00dc78c0688 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:57.694067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44616bf5-c227-48da-b45e-a00dc78c0688 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:57.711091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 628/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:33:57.717509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f14b89-94fc-49e2-aa6c-27cc63612f4a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:57.845381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:33:57.879759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f14b89-94fc-49e2-aa6c-27cc63612f4a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:57.924754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd596da-1b61-4904-8d5b-1da12f8e1684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:57.925508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd596da-1b61-4904-8d5b-1da12f8e1684 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:57.925794 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd596da-1b61-4904-8d5b-1da12f8e1684 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:57.926066 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd596da-1b61-4904-8d5b-1da12f8e1684 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:57.936216 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd596da-1b61-4904-8d5b-1da12f8e1684 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:57.938273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd596da-1b61-4904-8d5b-1da12f8e1684 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:57.954609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-ecd596da-1b61-4904-8d5b-1da12f8e1684 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:33:57.954609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:33:57.954609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:33:57.954609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:33:57.954609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:33:57.954609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:33:57.954609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:33:57.954609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:33:57.954609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:33:57.954609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:33:57.954609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:33:57.954609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:33:57.954609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:33:57.954609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:33:57.954609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:33:57.955825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:33:57.955825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:33:57.955825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:33:57.955825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:33:57.955825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:33:57.955825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:33:57.955825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:33:57.955825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:33:57.955825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:33:57.955825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:33:57.955825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:33:57.955825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:33:57.955825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:33:57.955825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:33:57.955825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:33:57.955825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:33:57.955825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:33:57.957049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:33:57.957049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:33:57.957049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:33:57.957049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:33:57.957049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:33:57.957049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:33:57.957049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:33:57.957049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:33:57.957049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 616/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:33:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:33:57.965164 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44616bf5-c227-48da-b45e-a00dc78c0688 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:58.120080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44616bf5-c227-48da-b45e-a00dc78c0688 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:58.164206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7503864-25b9-49af-8cb3-10acc7871ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:58.164919 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7503864-25b9-49af-8cb3-10acc7871ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:58.165470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7503864-25b9-49af-8cb3-10acc7871ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:58.165797 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7503864-25b9-49af-8cb3-10acc7871ec5 None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:58.166836 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7503864-25b9-49af-8cb3-10acc7871ec5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:58.167431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7503864-25b9-49af-8cb3-10acc7871ec5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:58.169808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7503864-25b9-49af-8cb3-10acc7871ec5 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:58.185847 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 629/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:33:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:33:58.193684 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd596da-1b61-4904-8d5b-1da12f8e1684 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:33:58.343817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd596da-1b61-4904-8d5b-1da12f8e1684 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:33:58.392666 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ffe1f1-a651-46aa-ad91-dbfbf5aedcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:33:58.393476 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffe1f1-a651-46aa-ad91-dbfbf5aedcd8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:33:58.393856 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffe1f1-a651-46aa-ad91-dbfbf5aedcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:33:58.394133 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffe1f1-a651-46aa-ad91-dbfbf5aedcd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:33:58.395194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ffe1f1-a651-46aa-ad91-dbfbf5aedcd8 None admin] RBAC: Adding query filter params (role.id=881210e567974dc0aa3043393c00a838, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=67fcd65eeb994b2381b83480e030f2ae) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:33:58.395615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ffe1f1-a651-46aa-ad91-dbfbf5aedcd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:33:58.397476 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ffe1f1-a651-46aa-ad91-dbfbf5aedcd8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:33:58.406401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 617/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:33:58 2022] GET /identity/v3/role_assignments?user.id=67fcd65eeb994b2381b83480e030f2ae&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=881210e567974dc0aa3043393c00a838 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:01.291142 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410e966-cc5c-4936-aa31-b8ad821a31e1 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:01.291758 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410e966-cc5c-4936-aa31-b8ad821a31e1 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:01.292012 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410e966-cc5c-4936-aa31-b8ad821a31e1 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:01.294492 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 630/1247] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:34:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:34:01.302392 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d3e31-d7e7-4730-ba96-121297013913 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:01.302971 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d3e31-d7e7-4730-ba96-121297013913 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:01.302971 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d3e31-d7e7-4730-ba96-121297013913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:01.310909 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:01.328037 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-cc1d3e31-d7e7-4730-ba96-121297013913 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:34:01.329595 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cc1d3e31-d7e7-4730-ba96-121297013913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:01.437587 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 618/1248] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:34:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:34:01.709463 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410e966-cc5c-4936-aa31-b8ad821a31e1 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:01.732630 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:01.843800 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410e966-cc5c-4936-aa31-b8ad821a31e1 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:01.882448 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b8c3f-db4a-4213-84b2-c3510b6ab062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:01.883135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b8c3f-db4a-4213-84b2-c3510b6ab062 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:01.883468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b8c3f-db4a-4213-84b2-c3510b6ab062 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:01.883816 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b8c3f-db4a-4213-84b2-c3510b6ab062 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:01.900137 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b8c3f-db4a-4213-84b2-c3510b6ab062 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:34:01.908699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-e24b8c3f-db4a-4213-84b2-c3510b6ab062 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:34:01.908699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:01.908699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:01.908699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:01.908699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:01.908699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:01.908699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:01.908699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:01.908699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:01.908699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:01.908699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:01.908699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:01.908699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:34:01.908699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:34:01.908699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:34:01.908699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:34:01.910007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:34:01.910007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:34:01.910007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:34:01.910007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:34:01.910007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:01.910007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:01.910007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:01.910007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:01.910007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:01.910007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:01.910007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:01.910007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:01.910007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:01.910007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:01.910007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:01.910007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:01.911155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:01.911155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:01.911155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:01.911155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:01.911155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:34:01.911155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:34:01.911155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:34:01.911155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:34:01.911155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:34:01.911155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:34:01.911155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:34:01.911155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:34:01.911155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 631/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:34:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:34:01.918840 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d3e31-d7e7-4730-ba96-121297013913 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:02.080547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d3e31-d7e7-4730-ba96-121297013913 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:02.114378 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:02.120865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394efd9-81e6-40a6-b71e-ae32154272e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:02.121658 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394efd9-81e6-40a6-b71e-ae32154272e7 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:02.121947 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394efd9-81e6-40a6-b71e-ae32154272e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:02.122213 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394efd9-81e6-40a6-b71e-ae32154272e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:02.122937 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394efd9-81e6-40a6-b71e-ae32154272e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:02.123232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394efd9-81e6-40a6-b71e-ae32154272e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:02.124631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394efd9-81e6-40a6-b71e-ae32154272e7 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:02.135641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 619/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:34:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:02.144909 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b8c3f-db4a-4213-84b2-c3510b6ab062 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:02.253081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:02.286979 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b8c3f-db4a-4213-84b2-c3510b6ab062 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:02.325970 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aec0d7-7abf-4e26-84f8-fe4561599774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:02.326703 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aec0d7-7abf-4e26-84f8-fe4561599774 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:02.327032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aec0d7-7abf-4e26-84f8-fe4561599774 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:02.327230 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aec0d7-7abf-4e26-84f8-fe4561599774 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:02.340129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aec0d7-7abf-4e26-84f8-fe4561599774 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:02.341741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aec0d7-7abf-4e26-84f8-fe4561599774 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:02.359993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-03aec0d7-7abf-4e26-84f8-fe4561599774 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:34:02.359993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:02.359993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:02.359993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:02.359993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:02.359993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:02.359993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:02.359993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:02.359993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:02.359993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:02.359993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:02.359993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:02.359993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:34:02.359993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:34:02.359993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:34:02.359993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:34:02.361097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:02.361097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:02.361097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:02.361097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:02.361097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:02.361097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:02.361097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:02.361097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:02.361097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:02.361097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:02.361097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:02.361097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:02.361097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:02.361097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:02.361097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:02.361097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:02.361097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:34:02.362073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:34:02.362073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:34:02.362073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:34:02.362073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 632/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:34:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:34:02.367392 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394efd9-81e6-40a6-b71e-ae32154272e7 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:02.497584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394efd9-81e6-40a6-b71e-ae32154272e7 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:02.536902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4947b-10c3-4c61-9192-998cab9b3f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:02.537558 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4947b-10c3-4c61-9192-998cab9b3f24 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:02.537838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4947b-10c3-4c61-9192-998cab9b3f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:02.538132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4947b-10c3-4c61-9192-998cab9b3f24 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:02.538945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4947b-10c3-4c61-9192-998cab9b3f24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:02.539259 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4947b-10c3-4c61-9192-998cab9b3f24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:02.540649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4947b-10c3-4c61-9192-998cab9b3f24 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:02.555766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 620/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:02.562108 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aec0d7-7abf-4e26-84f8-fe4561599774 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:02.693409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aec0d7-7abf-4e26-84f8-fe4561599774 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:02.744818 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:02.754742 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06c31f-ac95-422e-8090-91a498e49c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:02.755482 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06c31f-ac95-422e-8090-91a498e49c8c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:02.755780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06c31f-ac95-422e-8090-91a498e49c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:02.756090 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06c31f-ac95-422e-8090-91a498e49c8c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:02.765897 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06c31f-ac95-422e-8090-91a498e49c8c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:02.767647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06c31f-ac95-422e-8090-91a498e49c8c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:02.781099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-ae06c31f-ac95-422e-8090-91a498e49c8c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:34:02.781099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:02.781099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:02.781099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:02.781099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:02.781099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:02.781099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:02.781099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:02.781099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:02.781099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:02.781099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:02.781099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:02.781099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:34:02.781099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:34:02.781099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:34:02.781099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:02.782353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:02.783607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:02.783607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:02.783607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:34:02.783607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:34:02.783607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:34:02.783607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:34:02.783607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:34:02.783607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:34:02.783607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:34:02.783607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:34:02.783607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 633/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:34:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:34:02.791756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4947b-10c3-4c61-9192-998cab9b3f24 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:02.861949 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:02.934732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4947b-10c3-4c61-9192-998cab9b3f24 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:02.988098 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d382e-4735-4d8d-ba13-4ab3ca26cbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:02.988624 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d382e-4735-4d8d-ba13-4ab3ca26cbf3 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:02.988899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d382e-4735-4d8d-ba13-4ab3ca26cbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:02.989193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d382e-4735-4d8d-ba13-4ab3ca26cbf3 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:02.989932 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75d382e-4735-4d8d-ba13-4ab3ca26cbf3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:02.990231 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75d382e-4735-4d8d-ba13-4ab3ca26cbf3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:02.991760 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75d382e-4735-4d8d-ba13-4ab3ca26cbf3 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:03.020106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 621/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Sep 26 05:34:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:03.022081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06c31f-ac95-422e-8090-91a498e49c8c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:03.170709 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06c31f-ac95-422e-8090-91a498e49c8c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:03.219999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc2e57-8e20-48cc-8ad0-bd5e582f4385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:03.220652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc2e57-8e20-48cc-8ad0-bd5e582f4385 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:03.220932 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc2e57-8e20-48cc-8ad0-bd5e582f4385 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:03.221593 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc2e57-8e20-48cc-8ad0-bd5e582f4385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:03.316989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:03.338602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc2e57-8e20-48cc-8ad0-bd5e582f4385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:03.340992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc2e57-8e20-48cc-8ad0-bd5e582f4385 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:03.484825 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 634/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:34:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:34:03.491546 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d382e-4735-4d8d-ba13-4ab3ca26cbf3 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:03.616681 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d382e-4735-4d8d-ba13-4ab3ca26cbf3 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:03.631701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:03.656565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6031efd-50af-425e-b0d0-fa09cc3ccda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:03.657243 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6031efd-50af-425e-b0d0-fa09cc3ccda1 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:03.657522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6031efd-50af-425e-b0d0-fa09cc3ccda1 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:03.657798 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6031efd-50af-425e-b0d0-fa09cc3ccda1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:03.674018 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6031efd-50af-425e-b0d0-fa09cc3ccda1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:03.676290 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6031efd-50af-425e-b0d0-fa09cc3ccda1 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:03.694399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-a6031efd-50af-425e-b0d0-fa09cc3ccda1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:34:03.694399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:03.694399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:03.694399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:03.694399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:03.694399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:03.694399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:03.694399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:03.694399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:03.694399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:03.694399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:03.694399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:03.694399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:34:03.694399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:34:03.694399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:34:03.694399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:34:03.695468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:03.695468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:03.695468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:03.695468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:03.695468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:03.695468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:03.695468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:03.695468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:03.695468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:03.695468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:03.695468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:03.695468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:03.695468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:03.695468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:03.695468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:03.695468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:03.695468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:34:03.696435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:34:03.696435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:34:03.696435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:34:03.696435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 622/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:34:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:34:03.702138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc2e57-8e20-48cc-8ad0-bd5e582f4385 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:03.811112 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:03.825268 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc2e57-8e20-48cc-8ad0-bd5e582f4385 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:03.870248 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87697ed8-f6ed-414c-b398-b8200db63864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:03.870773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87697ed8-f6ed-414c-b398-b8200db63864 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:03.871048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87697ed8-f6ed-414c-b398-b8200db63864 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:03.871316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87697ed8-f6ed-414c-b398-b8200db63864 None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:03.871964 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87697ed8-f6ed-414c-b398-b8200db63864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:03.872256 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87697ed8-f6ed-414c-b398-b8200db63864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:03.873673 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87697ed8-f6ed-414c-b398-b8200db63864 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:03.886386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 635/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:03.892965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6031efd-50af-425e-b0d0-fa09cc3ccda1 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:04.019322 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6031efd-50af-425e-b0d0-fa09cc3ccda1 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:04.054912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98181e-bc93-4545-8b86-65509eef73f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:04.055536 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98181e-bc93-4545-8b86-65509eef73f6 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:04.055819 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98181e-bc93-4545-8b86-65509eef73f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:04.056081 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98181e-bc93-4545-8b86-65509eef73f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:04.063302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98181e-bc93-4545-8b86-65509eef73f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:04.064767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98181e-bc93-4545-8b86-65509eef73f6 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:04.077013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-ac98181e-bc93-4545-8b86-65509eef73f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:34:04.077013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:04.077013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:04.077013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:04.077013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:04.077013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:04.077013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:04.077013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:04.077013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:04.077013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:04.077013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:04.077013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:04.077013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:34:04.077013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:34:04.077013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:34:04.078179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:34:04.078179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:04.078179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:04.078179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:04.078179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:04.078179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:04.078179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:04.078179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:04.078179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:04.078179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:04.078179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:04.078179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:04.078179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:04.078179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:04.078179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:04.078179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:04.078179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:04.084204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:34:04.084204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:34:04.084204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:34:04.084204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:34:04.084204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:34:04.084204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:34:04.084204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:34:04.084204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:34:04.084204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 623/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:34:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:34:04.084826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87697ed8-f6ed-414c-b398-b8200db63864 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:04.186944 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87697ed8-f6ed-414c-b398-b8200db63864 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:04.230057 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10916dea-5963-4afd-b031-1cf0d097ed7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:04.230769 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10916dea-5963-4afd-b031-1cf0d097ed7f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:04.231069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10916dea-5963-4afd-b031-1cf0d097ed7f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:04.231360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10916dea-5963-4afd-b031-1cf0d097ed7f None admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:04.232131 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10916dea-5963-4afd-b031-1cf0d097ed7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:04.232468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10916dea-5963-4afd-b031-1cf0d097ed7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:04.234904 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10916dea-5963-4afd-b031-1cf0d097ed7f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:04.248744 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 636/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:34:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:04.256042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98181e-bc93-4545-8b86-65509eef73f6 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:04.284891 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:04.363862 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98181e-bc93-4545-8b86-65509eef73f6 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:04.399121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4741b7f-5add-42c7-9a78-d3b132c9d74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:04.399728 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4741b7f-5add-42c7-9a78-d3b132c9d74b None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:04.400005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4741b7f-5add-42c7-9a78-d3b132c9d74b None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:04.400272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4741b7f-5add-42c7-9a78-d3b132c9d74b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:04.401066 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4741b7f-5add-42c7-9a78-d3b132c9d74b None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=67fcd65eeb994b2381b83480e030f2ae) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:04.401402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4741b7f-5add-42c7-9a78-d3b132c9d74b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:04.402913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4741b7f-5add-42c7-9a78-d3b132c9d74b None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:04.409229 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 624/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:34:04 2022] GET /identity/v3/role_assignments?user.id=67fcd65eeb994b2381b83480e030f2ae&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:06.771525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8961bf-aa20-4ae1-a560-90e70badc4f9 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:06.772101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8961bf-aa20-4ae1-a560-90e70badc4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:06.772207 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8961bf-aa20-4ae1-a560-90e70badc4f9 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:06.773615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 637/1261] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:34:06.779861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4666e09-b08e-4805-af2f-0d168ab28ae5 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:06.780400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4666e09-b08e-4805-af2f-0d168ab28ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:06.780400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4666e09-b08e-4805-af2f-0d168ab28ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:06.800886 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d4666e09-b08e-4805-af2f-0d168ab28ae5 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:34:06.802318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d4666e09-b08e-4805-af2f-0d168ab28ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:06.880289 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 625/1262] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:34:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:34:07.187499 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8961bf-aa20-4ae1-a560-90e70badc4f9 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:07.200067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:07.308937 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8961bf-aa20-4ae1-a560-90e70badc4f9 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:07.350201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268543d-1f64-41c2-87a9-f2126f157b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:07.350831 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268543d-1f64-41c2-87a9-f2126f157b91 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:07.351257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268543d-1f64-41c2-87a9-f2126f157b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:07.352038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268543d-1f64-41c2-87a9-f2126f157b91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:07.362784 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3268543d-1f64-41c2-87a9-f2126f157b91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:34:07.367829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-3268543d-1f64-41c2-87a9-f2126f157b91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:34:07.367829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:07.367829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:07.367829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:07.367829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:07.367829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:07.367829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:07.367829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:07.367829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:07.367829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:07.367829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:07.367829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:07.367829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:34:07.367829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:34:07.367829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:34:07.367829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:34:07.368895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:34:07.368895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:34:07.368895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:34:07.368895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:34:07.368895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:07.368895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:07.368895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:07.368895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:07.368895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:07.368895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:07.368895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:07.368895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:07.368895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:07.368895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:07.368895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:07.368895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:07.369875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:07.369875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:07.369875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:07.369875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:07.369875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:34:07.369875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:34:07.369875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:34:07.369875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:34:07.369875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:34:07.369875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:34:07.369875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:34:07.369875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:34:07.369875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 638/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:34:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:34:07.375670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4666e09-b08e-4805-af2f-0d168ab28ae5 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:07.395467 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:07.488954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4666e09-b08e-4805-af2f-0d168ab28ae5 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:07.525233 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4473d-d186-4301-bada-5af14490ec68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:07.525897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4473d-d186-4301-bada-5af14490ec68 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:07.526177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4473d-d186-4301-bada-5af14490ec68 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:07.526446 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4473d-d186-4301-bada-5af14490ec68 None admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:07.527113 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4473d-d186-4301-bada-5af14490ec68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:07.527418 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4473d-d186-4301-bada-5af14490ec68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:07.528793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4473d-d186-4301-bada-5af14490ec68 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:07.537876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 626/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:34:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:07.543947 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268543d-1f64-41c2-87a9-f2126f157b91 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:07.615469 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:07.654029 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268543d-1f64-41c2-87a9-f2126f157b91 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:07.696239 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c74316-8319-4dba-b26c-70de46b264ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:07.696864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c74316-8319-4dba-b26c-70de46b264ca None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:07.697174 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c74316-8319-4dba-b26c-70de46b264ca None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:07.697436 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c74316-8319-4dba-b26c-70de46b264ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:07.708283 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c74316-8319-4dba-b26c-70de46b264ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:07.709767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c74316-8319-4dba-b26c-70de46b264ca None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:07.730824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-15c74316-8319-4dba-b26c-70de46b264ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:34:07.730824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:07.730824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:07.730824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:07.730824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:07.730824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:07.730824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:07.730824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:07.730824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:07.730824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:07.730824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:07.730824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:07.730824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:34:07.730824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:34:07.730824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:34:07.730824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:34:07.732153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:07.732153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:07.732153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:07.732153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:07.732153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:07.732153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:07.732153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:07.732153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:07.732153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:07.732153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:07.732153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:07.732153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:07.732153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:07.732153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:07.732153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:07.732153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:07.732153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:34:07.733420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:34:07.733420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:34:07.733420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:34:07.733420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 639/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:34:07.739314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4473d-d186-4301-bada-5af14490ec68 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:07.836760 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:07.862693 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4473d-d186-4301-bada-5af14490ec68 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:07.908478 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74052ba-2e8f-49f2-b98b-5a1f3d9d2e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:07.909313 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74052ba-2e8f-49f2-b98b-5a1f3d9d2e88 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:07.909739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74052ba-2e8f-49f2-b98b-5a1f3d9d2e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:07.910145 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74052ba-2e8f-49f2-b98b-5a1f3d9d2e88 None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:07.910805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74052ba-2e8f-49f2-b98b-5a1f3d9d2e88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:07.911161 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74052ba-2e8f-49f2-b98b-5a1f3d9d2e88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:07.912843 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74052ba-2e8f-49f2-b98b-5a1f3d9d2e88 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:07.932095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 627/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:07.938682 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c74316-8319-4dba-b26c-70de46b264ca None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:08.056556 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c74316-8319-4dba-b26c-70de46b264ca None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:08.077040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:08.093803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b647d6-1df8-42c0-b01c-29fe05a37390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:08.094450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b647d6-1df8-42c0-b01c-29fe05a37390 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:08.094778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b647d6-1df8-42c0-b01c-29fe05a37390 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:08.095085 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b647d6-1df8-42c0-b01c-29fe05a37390 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:08.104381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b647d6-1df8-42c0-b01c-29fe05a37390 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:08.105891 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b647d6-1df8-42c0-b01c-29fe05a37390 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:08.118025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-24b647d6-1df8-42c0-b01c-29fe05a37390 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:34:08.118025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:08.118025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:08.118025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:08.118025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:08.118025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:08.118025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:08.118025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:08.118025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:08.118025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:08.118025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:08.118025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:08.118025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:34:08.118025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:34:08.118025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:34:08.118025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:08.119213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:08.120411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:08.120411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:08.120411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:34:08.120411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:34:08.120411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:34:08.120411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:34:08.120411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:34:08.120411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:34:08.120411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:34:08.120411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:34:08.120411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 640/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:34:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:34:08.125797 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74052ba-2e8f-49f2-b98b-5a1f3d9d2e88 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:08.232882 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74052ba-2e8f-49f2-b98b-5a1f3d9d2e88 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:08.269184 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c4fdc-ea8f-470f-a8a4-58ce34aad6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:08.269819 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c4fdc-ea8f-470f-a8a4-58ce34aad6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:08.270095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c4fdc-ea8f-470f-a8a4-58ce34aad6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:08.270360 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c4fdc-ea8f-470f-a8a4-58ce34aad6e8 None admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:08.271097 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c4fdc-ea8f-470f-a8a4-58ce34aad6e8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:08.271398 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c4fdc-ea8f-470f-a8a4-58ce34aad6e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:08.272830 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c4fdc-ea8f-470f-a8a4-58ce34aad6e8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:08.282692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:08.294316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 628/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Sep 26 05:34:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:08.300838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b647d6-1df8-42c0-b01c-29fe05a37390 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:08.412828 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b647d6-1df8-42c0-b01c-29fe05a37390 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:08.452463 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc06911-f901-4a30-8146-84af9683846e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:08.453158 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc06911-f901-4a30-8146-84af9683846e None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:08.453450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc06911-f901-4a30-8146-84af9683846e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:08.453805 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc06911-f901-4a30-8146-84af9683846e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:08.465809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc06911-f901-4a30-8146-84af9683846e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:08.467515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc06911-f901-4a30-8146-84af9683846e None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:08.483013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-efc06911-f901-4a30-8146-84af9683846e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:34:08.483013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:08.483013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:08.483013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:08.483013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:08.483013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:08.483013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:08.483013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:08.483013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:08.483013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:08.483013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:08.483013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:08.483013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:34:08.483013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:34:08.483013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:34:08.483013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:34:08.484253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:08.484253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:08.484253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:08.484253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:08.484253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:08.484253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:08.484253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:08.484253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:08.484253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:08.484253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:08.484253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:08.484253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:08.484253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:08.484253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:08.484253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:08.484253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:08.484253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:34:08.485535 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:34:08.485535 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:34:08.485535 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:34:08.485535 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 641/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:34:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:34:08.490444 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c4fdc-ea8f-470f-a8a4-58ce34aad6e8 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:08.608949 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c4fdc-ea8f-470f-a8a4-58ce34aad6e8 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:08.645231 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49deb666-52f2-4518-a720-f86bf26ee34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:08.645838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deb666-52f2-4518-a720-f86bf26ee34a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:08.646114 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deb666-52f2-4518-a720-f86bf26ee34a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:08.646383 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49deb666-52f2-4518-a720-f86bf26ee34a None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:08.646997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49deb666-52f2-4518-a720-f86bf26ee34a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:08.647320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49deb666-52f2-4518-a720-f86bf26ee34a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:08.648931 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49deb666-52f2-4518-a720-f86bf26ee34a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:08.661107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 629/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:08.667379 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc06911-f901-4a30-8146-84af9683846e None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:08.700108 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:08.780224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc06911-f901-4a30-8146-84af9683846e None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:08.819411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b54194-d106-44a8-b268-380e7c4408da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:08.820046 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b54194-d106-44a8-b268-380e7c4408da None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:08.820315 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b54194-d106-44a8-b268-380e7c4408da None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:08.820576 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b54194-d106-44a8-b268-380e7c4408da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:08.827734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b54194-d106-44a8-b268-380e7c4408da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:08.829311 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b54194-d106-44a8-b268-380e7c4408da None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:08.841568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-e0b54194-d106-44a8-b268-380e7c4408da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:34:08.841568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:08.841568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:08.841568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:08.841568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:08.841568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:08.841568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:08.841568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:08.841568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:08.841568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:08.841568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:08.841568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:08.841568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:34:08.841568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:34:08.841568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:34:08.842873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:34:08.842873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:08.842873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:08.842873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:08.842873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:08.842873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:08.842873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:08.842873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:08.842873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:08.842873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:08.842873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:08.842873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:08.842873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:08.842873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:08.842873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:08.842873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:08.842873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:08.843914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:34:08.843914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:34:08.843914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:34:08.843914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:34:08.843914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:34:08.843914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:34:08.843914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:34:08.843914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:34:08.843914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 642/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:34:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 05:34:08.849446 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49deb666-52f2-4518-a720-f86bf26ee34a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:08.943825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:08.995963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49deb666-52f2-4518-a720-f86bf26ee34a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:09.031084 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ddbff-f473-4fd7-a2b5-8744c59b6a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:09.031725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ddbff-f473-4fd7-a2b5-8744c59b6a1e None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:09.032006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ddbff-f473-4fd7-a2b5-8744c59b6a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:09.032325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ddbff-f473-4fd7-a2b5-8744c59b6a1e None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:09.033023 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ddbff-f473-4fd7-a2b5-8744c59b6a1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:09.033397 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ddbff-f473-4fd7-a2b5-8744c59b6a1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:09.034936 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ddbff-f473-4fd7-a2b5-8744c59b6a1e None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:09.048266 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 630/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:34:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:09.055061 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b54194-d106-44a8-b268-380e7c4408da None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:09.150323 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:09.160080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b54194-d106-44a8-b268-380e7c4408da None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:09.200297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce897060-8fd5-49ac-9908-3ec93465d638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:09.200987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce897060-8fd5-49ac-9908-3ec93465d638 None admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:09.201305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce897060-8fd5-49ac-9908-3ec93465d638 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:09.201588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce897060-8fd5-49ac-9908-3ec93465d638 None admin] PATH_INFO: `/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/67fcd65eeb994b2381b83480e030f2ae/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:09.204091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce897060-8fd5-49ac-9908-3ec93465d638 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e44d601292b47b7b66d6f3efa80b450, user_id=67fcd65eeb994b2381b83480e030f2ae, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:09.205590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce897060-8fd5-49ac-9908-3ec93465d638 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:09.222323 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 643/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Mon Sep 26 05:34:09 2022] PUT /identity/v3/projects/5e44d601292b47b7b66d6f3efa80b450/users/67fcd65eeb994b2381b83480e030f2ae/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:34:11.483831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd9073-dd60-4964-8f0c-5f62539747ca None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:11.484528 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd9073-dd60-4964-8f0c-5f62539747ca None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:11.484528 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd9073-dd60-4964-8f0c-5f62539747ca None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:11.486100 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 631/1274] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:34:11.491767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e77da-89e6-41fb-802e-04b7dad3dde9 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:11.492369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e77da-89e6-41fb-802e-04b7dad3dde9 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:11.492531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e77da-89e6-41fb-802e-04b7dad3dde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:11.515934 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-0f4e77da-89e6-41fb-802e-04b7dad3dde9 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:34:11.517679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-0f4e77da-89e6-41fb-802e-04b7dad3dde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:11.607604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 644/1275] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:34:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:34:11.882890 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd9073-dd60-4964-8f0c-5f62539747ca None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:12.009334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd9073-dd60-4964-8f0c-5f62539747ca None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:12.031303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:12.055355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e7c9a-741b-42ae-9573-61ecc6164bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:12.056412 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e7c9a-741b-42ae-9573-61ecc6164bae None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:12.056908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e7c9a-741b-42ae-9573-61ecc6164bae None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:12.057486 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e7c9a-741b-42ae-9573-61ecc6164bae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:12.076718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e7c9a-741b-42ae-9573-61ecc6164bae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 05:34:12.086674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-3d0e7c9a-741b-42ae-9573-61ecc6164bae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:34:12.086674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:12.086674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:12.086674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:12.086674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:12.086674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:12.086674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:12.086674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:12.086674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:12.086674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:12.086674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:12.086674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:12.086674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 05:34:12.086674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 05:34:12.086674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 05:34:12.086674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 05:34:12.088132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 05:34:12.088132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 05:34:12.088132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 05:34:12.088132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 05:34:12.088132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:12.088132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:12.088132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:12.088132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:12.088132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:12.088132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:12.088132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:12.088132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:12.088132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:12.088132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:12.088132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:12.088132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:12.089556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:12.089556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:12.089556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:12.089556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:12.089556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 05:34:12.089556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 05:34:12.089556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 05:34:12.089556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 05:34:12.089556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 05:34:12.089556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 05:34:12.089556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 05:34:12.089556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:34:12.089556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 632/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Mon Sep 26 05:34:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:34:12.095132 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e77da-89e6-41fb-802e-04b7dad3dde9 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:12.198780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:12.222855 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e77da-89e6-41fb-802e-04b7dad3dde9 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:12.259774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50b8b1-0212-40b6-a723-01c4196a362f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:12.260627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50b8b1-0212-40b6-a723-01c4196a362f None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:12.260778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50b8b1-0212-40b6-a723-01c4196a362f None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:12.260994 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50b8b1-0212-40b6-a723-01c4196a362f None admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:12.261813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50b8b1-0212-40b6-a723-01c4196a362f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:12.262250 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50b8b1-0212-40b6-a723-01c4196a362f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:12.263761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50b8b1-0212-40b6-a723-01c4196a362f None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:12.273627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 645/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Mon Sep 26 05:34:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:12.280756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e7c9a-741b-42ae-9573-61ecc6164bae None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:12.418923 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e7c9a-741b-42ae-9573-61ecc6164bae None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:12.460868 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f457129-d529-4c25-9c8e-af4e7efc1846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:12.461555 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f457129-d529-4c25-9c8e-af4e7efc1846 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:12.461859 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f457129-d529-4c25-9c8e-af4e7efc1846 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:12.462136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f457129-d529-4c25-9c8e-af4e7efc1846 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:12.476119 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f457129-d529-4c25-9c8e-af4e7efc1846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:12.477802 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f457129-d529-4c25-9c8e-af4e7efc1846 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:12.496224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-7f457129-d529-4c25-9c8e-af4e7efc1846 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:34:12.496224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:12.496224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:12.496224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:12.496224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:12.496224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:12.496224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:12.496224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:12.496224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:12.496224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:12.496224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:12.496224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:12.496224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:34:12.496224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:34:12.496224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:34:12.496224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:34:12.497899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:12.497899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:12.497899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:12.497899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:12.497899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:12.497899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:12.497899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:12.497899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:12.497899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:12.497899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:12.497899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:12.497899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:12.497899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:12.497899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:12.497899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:12.497899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:12.497899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:34:12.498982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:34:12.498982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:34:12.498982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:34:12.498982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 633/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:34:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:34:12.503581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50b8b1-0212-40b6-a723-01c4196a362f None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:12.610286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50b8b1-0212-40b6-a723-01c4196a362f None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:12.647348 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89d274-039c-4acb-a0d6-fdb1f0086fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:12.647969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89d274-039c-4acb-a0d6-fdb1f0086fce None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:12.648250 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89d274-039c-4acb-a0d6-fdb1f0086fce None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:12.648523 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89d274-039c-4acb-a0d6-fdb1f0086fce None admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:12.649199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89d274-039c-4acb-a0d6-fdb1f0086fce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:12.649507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89d274-039c-4acb-a0d6-fdb1f0086fce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:12.650986 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89d274-039c-4acb-a0d6-fdb1f0086fce None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:12.663662 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 646/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:34:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:12.670012 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f457129-d529-4c25-9c8e-af4e7efc1846 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:12.708324 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:12.793659 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f457129-d529-4c25-9c8e-af4e7efc1846 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:12.847383 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce44ef-99da-423d-9be8-5bc46a21167a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:12.848048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce44ef-99da-423d-9be8-5bc46a21167a None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:12.848307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce44ef-99da-423d-9be8-5bc46a21167a None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:12.848574 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce44ef-99da-423d-9be8-5bc46a21167a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:12.857896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce44ef-99da-423d-9be8-5bc46a21167a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:12.859383 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce44ef-99da-423d-9be8-5bc46a21167a None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:12.872928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-79ce44ef-99da-423d-9be8-5bc46a21167a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:34:12.872928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:12.872928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:12.872928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:12.872928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:12.872928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:12.872928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:12.872928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:12.872928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:12.872928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:12.872928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:12.872928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:12.872928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 05:34:12.872928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 05:34:12.872928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 05:34:12.872928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:12.874124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:12.875082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:12.875082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:12.875082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 05:34:12.875082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 05:34:12.875082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 05:34:12.875082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 05:34:12.875082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 05:34:12.875082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 05:34:12.875082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 05:34:12.875082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:34:12.875082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 634/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:34:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:34:12.886450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89d274-039c-4acb-a0d6-fdb1f0086fce None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:12.899502 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:13.024216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89d274-039c-4acb-a0d6-fdb1f0086fce None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:13.066777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbf835-ab05-449f-ad2b-b3139c15d1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:13.067569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbf835-ab05-449f-ad2b-b3139c15d1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:13.068024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbf835-ab05-449f-ad2b-b3139c15d1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:13.068418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbf835-ab05-449f-ad2b-b3139c15d1d9 None admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:13.069296 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adbf835-ab05-449f-ad2b-b3139c15d1d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:13.069666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adbf835-ab05-449f-ad2b-b3139c15d1d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:13.071842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adbf835-ab05-449f-ad2b-b3139c15d1d9 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:13.093539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 647/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Mon Sep 26 05:34:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:13.100181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce44ef-99da-423d-9be8-5bc46a21167a None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:13.218695 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:13.233635 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce44ef-99da-423d-9be8-5bc46a21167a None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:13.270577 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c356686-9cc8-4cd0-98df-075b765c34ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:13.271253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c356686-9cc8-4cd0-98df-075b765c34ee None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:13.271570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c356686-9cc8-4cd0-98df-075b765c34ee None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:13.271837 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c356686-9cc8-4cd0-98df-075b765c34ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:13.353734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c356686-9cc8-4cd0-98df-075b765c34ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:13.355345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c356686-9cc8-4cd0-98df-075b765c34ee None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:13.462233 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:13.474151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 635/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Mon Sep 26 05:34:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 05:34:13.480384 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbf835-ab05-449f-ad2b-b3139c15d1d9 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:13.552329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:13.596584 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbf835-ab05-449f-ad2b-b3139c15d1d9 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:13.633958 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed07fc1-8ffc-4657-a9f4-e57358df6309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:13.634598 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed07fc1-8ffc-4657-a9f4-e57358df6309 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:13.634931 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed07fc1-8ffc-4657-a9f4-e57358df6309 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:13.635368 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed07fc1-8ffc-4657-a9f4-e57358df6309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:13.648958 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed07fc1-8ffc-4657-a9f4-e57358df6309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:13.650588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed07fc1-8ffc-4657-a9f4-e57358df6309 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:13.668140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-9ed07fc1-8ffc-4657-a9f4-e57358df6309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:34:13.668140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:13.668140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:13.668140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:13.668140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:13.668140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:13.668140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:13.668140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:13.668140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:13.668140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:13.668140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:13.668140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:13.668140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 05:34:13.668140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 05:34:13.668140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 05:34:13.668140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 05:34:13.669390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:13.669390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:13.669390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:13.669390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:13.669390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:13.669390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:13.669390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:13.669390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:13.669390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:13.669390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:13.669390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:13.669390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:13.669390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:13.669390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:13.669390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:13.669390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:13.669390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 05:34:13.670501 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 05:34:13.670501 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 05:34:13.670501 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:34:13.670501 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 648/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:34:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:34:13.676052 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c356686-9cc8-4cd0-98df-075b765c34ee None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:13.797536 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c356686-9cc8-4cd0-98df-075b765c34ee None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:13.837771 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21a0ca-a193-4709-94ef-e0d51649daaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:13.838482 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21a0ca-a193-4709-94ef-e0d51649daaa None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:13.838758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21a0ca-a193-4709-94ef-e0d51649daaa None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:13.839147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21a0ca-a193-4709-94ef-e0d51649daaa None admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:13.839827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21a0ca-a193-4709-94ef-e0d51649daaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:13.840154 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21a0ca-a193-4709-94ef-e0d51649daaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:13.841730 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21a0ca-a193-4709-94ef-e0d51649daaa None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:13.856267 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 636/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Mon Sep 26 05:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:13.862166 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed07fc1-8ffc-4657-a9f4-e57358df6309 None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:13.972187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed07fc1-8ffc-4657-a9f4-e57358df6309 None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:13.986816 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:14.012476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8517510-8fd4-4116-8392-2a85e52012cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:14.013179 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8517510-8fd4-4116-8392-2a85e52012cd None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:14.013457 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8517510-8fd4-4116-8392-2a85e52012cd None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:14.013710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8517510-8fd4-4116-8392-2a85e52012cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:14.021567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8517510-8fd4-4116-8392-2a85e52012cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:14.023106 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8517510-8fd4-4116-8392-2a85e52012cd None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:14.035355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application [None req-f8517510-8fd4-4116-8392-2a85e52012cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:34:14.035355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:14.035355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:14.035355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:14.035355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:14.035355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:14.035355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:14.035355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:14.035355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:14.035355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:14.035355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:14.035355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:14.035355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 05:34:14.035355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 05:34:14.035355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 05:34:14.036356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 05:34:14.036356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:14.036356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:14.036356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:14.036356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:14.036356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:14.036356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:14.036356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:14.036356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:14.036356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:14.036356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:14.036356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:14.036356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:14.036356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:14.036356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:14.036356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:14.036356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:14.037357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 05:34:14.037357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 05:34:14.037357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 05:34:14.037357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 05:34:14.037357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 05:34:14.037357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 05:34:14.037357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 05:34:14.037357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: ERROR keystone.server.flask.application Sep 26 05:34:14.037357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 649/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Mon Sep 26 05:34:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:34:14.042922 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21a0ca-a193-4709-94ef-e0d51649daaa None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:14.098253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:14.155244 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21a0ca-a193-4709-94ef-e0d51649daaa None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:14.191146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7d65a-9fc8-47c3-b643-bdea6ed15a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:14.191753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7d65a-9fc8-47c3-b643-bdea6ed15a83 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:14.192026 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7d65a-9fc8-47c3-b643-bdea6ed15a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:14.192296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7d65a-9fc8-47c3-b643-bdea6ed15a83 None admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:14.192985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7d65a-9fc8-47c3-b643-bdea6ed15a83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:14.193318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7d65a-9fc8-47c3-b643-bdea6ed15a83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:14.194750 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7d65a-9fc8-47c3-b643-bdea6ed15a83 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:14.208932 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 637/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Mon Sep 26 05:34:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:14.215721 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8517510-8fd4-4116-8392-2a85e52012cd None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:14.342921 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8517510-8fd4-4116-8392-2a85e52012cd None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:14.378764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c226750-ee75-4462-b482-89eef2e61326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:14.379371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c226750-ee75-4462-b482-89eef2e61326 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:14.379648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c226750-ee75-4462-b482-89eef2e61326 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:14.379911 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c226750-ee75-4462-b482-89eef2e61326 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:14.380684 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c226750-ee75-4462-b482-89eef2e61326 None admin] RBAC: Adding query filter params (role.id=4caa7fc301d3427789d438f4c8937d7a, scope.project.id=5e44d601292b47b7b66d6f3efa80b450, user.id=67fcd65eeb994b2381b83480e030f2ae) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:14.380989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c226750-ee75-4462-b482-89eef2e61326 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:14.382507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c226750-ee75-4462-b482-89eef2e61326 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:14.389209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 650/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Mon Sep 26 05:34:14 2022] GET /identity/v3/role_assignments?user.id=67fcd65eeb994b2381b83480e030f2ae&scope.project.id=5e44d601292b47b7b66d6f3efa80b450&role.id=4caa7fc301d3427789d438f4c8937d7a => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:16.686978 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a8238-6fbc-4408-9b18-c69ae7e7d7de None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:16.687547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a8238-6fbc-4408-9b18-c69ae7e7d7de None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:16.687547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a8238-6fbc-4408-9b18-c69ae7e7d7de None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:16.688894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 638/1288] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:34:16.694978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417c88b-888b-4910-b305-b0da03438735 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:16.695462 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417c88b-888b-4910-b305-b0da03438735 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:16.695556 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417c88b-888b-4910-b305-b0da03438735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:16.715432 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-2417c88b-888b-4910-b305-b0da03438735 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:34:16.716673 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2417c88b-888b-4910-b305-b0da03438735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:16.749396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:16.789978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 651/1289] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:34:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:34:17.083596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a8238-6fbc-4408-9b18-c69ae7e7d7de None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:17.199001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a8238-6fbc-4408-9b18-c69ae7e7d7de None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:17.210656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:17.237806 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571238d-8aa3-4de2-b869-beb8fa9b1c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:17.238449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571238d-8aa3-4de2-b869-beb8fa9b1c96 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:17.238723 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571238d-8aa3-4de2-b869-beb8fa9b1c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:17.238987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571238d-8aa3-4de2-b869-beb8fa9b1c96 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:17.239574 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571238d-8aa3-4de2-b869-beb8fa9b1c96 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:17.240955 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571238d-8aa3-4de2-b869-beb8fa9b1c96 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:17.250790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-5571238d-8aa3-4de2-b869-beb8fa9b1c96 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:34:17.250790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:17.250790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:34:17.250790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:34:17.250790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:34:17.250790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:34:17.250790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:34:17.250790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:34:17.250790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:34:17.250790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:34:17.250790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:34:17.250790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:34:17.250790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:17.250790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:17.250790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:17.250790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:17.250790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:17.252127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:17.252127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:17.252127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:17.252127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:17.252127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:17.252127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:17.252127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:34:17.252127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:34:17.252127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:34:17.252127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:34:17.252127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:17.252127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:17.252127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:17.252127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:17.252127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:17.252127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:17.253303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:17.253303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:17.253303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:17.253303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:17.253303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:17.253303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:17.253303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:17.253303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:17.253303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:17.253303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:17.253303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:34:17.253303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:34:17.253303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:34:17.253303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:34:17.253303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 639/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 26 05:34:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:34:17.258462 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417c88b-888b-4910-b305-b0da03438735 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:17.382469 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417c88b-888b-4910-b305-b0da03438735 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:17.422127 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce43518-a80b-4f41-b0e3-63d5cae3a103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:17.422737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce43518-a80b-4f41-b0e3-63d5cae3a103 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:17.422992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce43518-a80b-4f41-b0e3-63d5cae3a103 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:17.423271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce43518-a80b-4f41-b0e3-63d5cae3a103 None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:17.423900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce43518-a80b-4f41-b0e3-63d5cae3a103 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:17.424178 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce43518-a80b-4f41-b0e3-63d5cae3a103 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:17.425592 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce43518-a80b-4f41-b0e3-63d5cae3a103 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:17.431201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 652/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Sep 26 05:34:17 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:34:17.437567 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571238d-8aa3-4de2-b869-beb8fa9b1c96 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:17.547918 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571238d-8aa3-4de2-b869-beb8fa9b1c96 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:17.599459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f4662-5d21-47a1-9ea4-d7b361705db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:17.600328 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f4662-5d21-47a1-9ea4-d7b361705db8 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:17.600809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f4662-5d21-47a1-9ea4-d7b361705db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:17.601330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f4662-5d21-47a1-9ea4-d7b361705db8 None admin] PATH_INFO: `/v3/services` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:17.602492 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f4662-5d21-47a1-9ea4-d7b361705db8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:17.603020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f4662-5d21-47a1-9ea4-d7b361705db8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:17.605711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f4662-5d21-47a1-9ea4-d7b361705db8 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:17.613941 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 640/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Sep 26 05:34:17 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:19.795184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a64f6-2495-4aae-8a0f-04f35a2f3bcd None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:19.795782 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a64f6-2495-4aae-8a0f-04f35a2f3bcd None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:19.795782 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a64f6-2495-4aae-8a0f-04f35a2f3bcd None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:19.797064 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 653/1293] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:34:19.802453 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297feeb6-67e9-4ae8-b9be-ce6f636f941f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:19.802890 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297feeb6-67e9-4ae8-b9be-ce6f636f941f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:19.802983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297feeb6-67e9-4ae8-b9be-ce6f636f941f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:19.822301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-297feeb6-67e9-4ae8-b9be-ce6f636f941f None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:34:19.823601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-297feeb6-67e9-4ae8-b9be-ce6f636f941f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:19.855212 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:19.895737 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 641/1294] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:34:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:34:20.158816 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a64f6-2495-4aae-8a0f-04f35a2f3bcd None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:20.193593 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:20.269664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a64f6-2495-4aae-8a0f-04f35a2f3bcd None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:20.308031 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c9b89-222f-4410-9918-4d776fb6bbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:20.308641 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c9b89-222f-4410-9918-4d776fb6bbc2 None admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:20.308916 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c9b89-222f-4410-9918-4d776fb6bbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:20.309215 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c9b89-222f-4410-9918-4d776fb6bbc2 None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:20.309890 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c9b89-222f-4410-9918-4d776fb6bbc2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:20.311239 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c9b89-222f-4410-9918-4d776fb6bbc2 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:20.326136 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 654/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Mon Sep 26 05:34:20 2022] POST /identity/v3/services => generated 246 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:34:22.493267 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75729ca3-ff57-4623-8b06-be7ee9c90684 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:22.493824 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75729ca3-ff57-4623-8b06-be7ee9c90684 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:22.493824 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75729ca3-ff57-4623-8b06-be7ee9c90684 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:22.495187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 642/1296] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:34:22.501152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1aec82-0966-42db-ac0e-aaaebb59daea None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:22.501614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1aec82-0966-42db-ac0e-aaaebb59daea None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:22.501950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1aec82-0966-42db-ac0e-aaaebb59daea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:22.527629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-df1aec82-0966-42db-ac0e-aaaebb59daea None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:34:22.528842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-df1aec82-0966-42db-ac0e-aaaebb59daea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:22.602911 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 655/1297] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:34:22 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:34:22.871805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75729ca3-ff57-4623-8b06-be7ee9c90684 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:22.981919 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75729ca3-ff57-4623-8b06-be7ee9c90684 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:23.017761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3065cd-fb4d-4b81-8dae-270051e3bc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:23.018387 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3065cd-fb4d-4b81-8dae-270051e3bc4c None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:23.018665 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3065cd-fb4d-4b81-8dae-270051e3bc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:23.018934 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3065cd-fb4d-4b81-8dae-270051e3bc4c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:23.019523 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3065cd-fb4d-4b81-8dae-270051e3bc4c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:23.020888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3065cd-fb4d-4b81-8dae-270051e3bc4c None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:23.030118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-8e3065cd-fb4d-4b81-8dae-270051e3bc4c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:34:23.030118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:23.030118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:34:23.030118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:34:23.030118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:34:23.030118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:34:23.030118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:34:23.030118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:34:23.030118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:34:23.030118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:34:23.030118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:34:23.030118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:34:23.030118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:23.030118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:23.030118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:23.030118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:23.030118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:23.031254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:23.031254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:23.031254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:23.031254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:23.031254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:23.031254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:23.031254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:34:23.031254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:34:23.031254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:34:23.031254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:34:23.031254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:23.031254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:23.031254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:23.031254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:23.031254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:23.031254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:23.032105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:23.032105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:23.032105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:23.032105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:23.032105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:23.032105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:23.032105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:23.032105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:23.032105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:23.032105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:23.032105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:34:23.032105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:34:23.032105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:34:23.032105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:34:23.032105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 643/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 26 05:34:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:34:23.037672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1aec82-0966-42db-ac0e-aaaebb59daea None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:23.057672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:23.142221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1aec82-0966-42db-ac0e-aaaebb59daea None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:23.176752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08094476-2be8-4a68-bfe3-07ded773e64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:23.177384 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08094476-2be8-4a68-bfe3-07ded773e64c None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:23.177659 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08094476-2be8-4a68-bfe3-07ded773e64c None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:23.177928 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08094476-2be8-4a68-bfe3-07ded773e64c None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:23.178549 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08094476-2be8-4a68-bfe3-07ded773e64c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:23.178843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08094476-2be8-4a68-bfe3-07ded773e64c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:23.180199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08094476-2be8-4a68-bfe3-07ded773e64c None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:23.185515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 656/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Sep 26 05:34:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:23.192016 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3065cd-fb4d-4b81-8dae-270051e3bc4c None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:23.227066 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:23.300875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3065cd-fb4d-4b81-8dae-270051e3bc4c None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:23.338436 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b80a52-9dd2-4497-8ee4-cd2d92f70502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:23.339079 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b80a52-9dd2-4497-8ee4-cd2d92f70502 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:23.339362 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b80a52-9dd2-4497-8ee4-cd2d92f70502 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:23.339623 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b80a52-9dd2-4497-8ee4-cd2d92f70502 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:23.340327 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b80a52-9dd2-4497-8ee4-cd2d92f70502 None admin] RBAC: Adding query filter params (interface=public, service_id=4215a0d227a446b2aa407db473744290, region_id=RegionOne) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:23.340637 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b80a52-9dd2-4497-8ee4-cd2d92f70502 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:23.342078 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b80a52-9dd2-4497-8ee4-cd2d92f70502 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:23.347516 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 644/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Mon Sep 26 05:34:23 2022] GET /identity/v3/endpoints?service_id=4215a0d227a446b2aa407db473744290&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:23.353536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08094476-2be8-4a68-bfe3-07ded773e64c None admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:23.437912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:23.458492 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08094476-2be8-4a68-bfe3-07ded773e64c None admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:23.496807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be403cc-ba46-4372-ac0a-065ce3e972c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:23.497440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be403cc-ba46-4372-ac0a-065ce3e972c5 None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:23.497722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be403cc-ba46-4372-ac0a-065ce3e972c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:23.497990 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be403cc-ba46-4372-ac0a-065ce3e972c5 None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:23.498584 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be403cc-ba46-4372-ac0a-065ce3e972c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:23.499937 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be403cc-ba46-4372-ac0a-065ce3e972c5 None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:23.506608 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 657/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Mon Sep 26 05:34:23 2022] GET /identity/v3/services => generated 2191 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:34:25.608759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1dac7-9927-4cf1-9e8a-238b531090fe None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:25.609408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1dac7-9927-4cf1-9e8a-238b531090fe None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:25.609408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1dac7-9927-4cf1-9e8a-238b531090fe None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:25.610743 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 645/1302] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:34:25.616437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bbbac-ea75-492d-899f-7dc6d8c52585 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:25.616883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bbbac-ea75-492d-899f-7dc6d8c52585 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:25.617033 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bbbac-ea75-492d-899f-7dc6d8c52585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:25.636567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-580bbbac-ea75-492d-899f-7dc6d8c52585 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:34:25.637800 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-580bbbac-ea75-492d-899f-7dc6d8c52585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:34:25.709459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 658/1303] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:34:25 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 05:34:25.970628 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1dac7-9927-4cf1-9e8a-238b531090fe None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:26.080121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:26.087808 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1dac7-9927-4cf1-9e8a-238b531090fe None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:26.124606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f5468-7dc4-4e37-87ae-6db8a868f003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:26.125269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f5468-7dc4-4e37-87ae-6db8a868f003 None admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:26.125669 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f5468-7dc4-4e37-87ae-6db8a868f003 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:26.125939 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f5468-7dc4-4e37-87ae-6db8a868f003 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:26.126524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f5468-7dc4-4e37-87ae-6db8a868f003 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:26.127899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f5468-7dc4-4e37-87ae-6db8a868f003 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:26.137087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application [None req-782f5468-7dc4-4e37-87ae-6db8a868f003 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:34:26.137087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:26.137087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 05:34:26.137087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 05:34:26.137087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 05:34:26.137087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 05:34:26.137087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 05:34:26.137087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:34:26.137087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:34:26.137087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:34:26.137087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 05:34:26.137087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:34:26.137087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 05:34:26.137087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 05:34:26.137087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 05:34:26.137087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 05:34:26.137087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 05:34:26.138323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 05:34:26.138323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 05:34:26.138323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 05:34:26.138323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 05:34:26.138323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 05:34:26.138323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 05:34:26.138323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 05:34:26.138323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 05:34:26.138323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 05:34:26.138323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 05:34:26.138323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 05:34:26.138323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 05:34:26.138323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 05:34:26.138323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 05:34:26.138323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 05:34:26.138323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 05:34:26.139395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 05:34:26.139395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application with Lock( Sep 26 05:34:26.139395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 05:34:26.139395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self._enter() Sep 26 05:34:26.139395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 05:34:26.139395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 05:34:26.139395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 05:34:26.139395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application return self.creator() Sep 26 05:34:26.139395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 05:34:26.139395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application created_value = creator( Sep 26 05:34:26.139395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 05:34:26.139395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 05:34:26.139395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 05:34:26.139395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: ERROR keystone.server.flask.application Sep 26 05:34:26.139395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 646/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Mon Sep 26 05:34:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 05:34:26.144744 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bbbac-ea75-492d-899f-7dc6d8c52585 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:26.224861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:26.250699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bbbac-ea75-492d-899f-7dc6d8c52585 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:26.286773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427180a-c2ff-44fc-9668-03e98aa0eb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:26.287454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427180a-c2ff-44fc-9668-03e98aa0eb1e None admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:26.287804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427180a-c2ff-44fc-9668-03e98aa0eb1e None admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:26.288140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427180a-c2ff-44fc-9668-03e98aa0eb1e None admin] PATH_INFO: `/v3/services` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:26.288812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427180a-c2ff-44fc-9668-03e98aa0eb1e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:34:26.289162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427180a-c2ff-44fc-9668-03e98aa0eb1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:26.290722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427180a-c2ff-44fc-9668-03e98aa0eb1e None admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:26.296934 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 659/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Mon Sep 26 05:34:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:34:26.303267 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f5468-7dc4-4e37-87ae-6db8a868f003 None admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:34:26.408412 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f5468-7dc4-4e37-87ae-6db8a868f003 None admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:34:26.444236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34b118-00b1-4295-859f-d4a9cc943b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:34:26.444844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34b118-00b1-4295-859f-d4a9cc943b79 None admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:34:26.445122 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34b118-00b1-4295-859f-d4a9cc943b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:34:26.445418 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34b118-00b1-4295-859f-d4a9cc943b79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:34:26.446084 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34b118-00b1-4295-859f-d4a9cc943b79 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:34:26.447460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34b118-00b1-4295-859f-d4a9cc943b79 None admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:34:26.472272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 647/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:34:26 2022] POST /identity/v3/endpoints => generated 331 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:34:36.293285 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=30 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:34:36.466950 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:33.178649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad929885-06b4-43dc-8fd6-c8b11dd12711 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:33.179131 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad929885-06b4-43dc-8fd6-c8b11dd12711 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:33.179218 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad929885-06b4-43dc-8fd6-c8b11dd12711 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:33.186621 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d350c-70a9-421e-b5ed-c6c3ddb05b82 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:33.187088 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d350c-70a9-421e-b5ed-c6c3ddb05b82 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:33.187169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d350c-70a9-421e-b5ed-c6c3ddb05b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:33.192157 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 660/1307] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:35:33 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:35:33.288891 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-df6d350c-70a9-421e-b5ed-c6c3ddb05b82 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:33.290505 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-df6d350c-70a9-421e-b5ed-c6c3ddb05b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:33.404639 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 648/1308] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:35:33.764358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda638f-3a8d-4844-b082-8ab0169f4c91 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:33.765050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda638f-3a8d-4844-b082-8ab0169f4c91 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:33.765573 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda638f-3a8d-4844-b082-8ab0169f4c91 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:33.767530 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 661/1309] 10.222.0.57 () {54 vars in 964 bytes} [Mon Sep 26 05:35:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:35:33.775424 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ca2cb-49b9-4155-9775-bbb2d748e5d0 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:33.775766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ca2cb-49b9-4155-9775-bbb2d748e5d0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:33.776215 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ca2cb-49b9-4155-9775-bbb2d748e5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:33.886257 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-3b5ca2cb-49b9-4155-9775-bbb2d748e5d0 None None] MFA Rules not processed for user `6bf0f594ce7646b7b6ddb8a8553509d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:33.888043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3b5ca2cb-49b9-4155-9775-bbb2d748e5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:33.914857 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:33.997024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda638f-3a8d-4844-b082-8ab0169f4c91 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:33.999393 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-cdda638f-3a8d-4844-b082-8ab0169f4c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:34.004158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 649/1310] 10.222.0.57 () {58 vars in 1061 bytes} [Mon Sep 26 05:35:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:34.135991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda638f-3a8d-4844-b082-8ab0169f4c91 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:34.231694 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f6f8c-44ae-477f-a8a8-c8033de7f7cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:34.232481 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f6f8c-44ae-477f-a8a8-c8033de7f7cd service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:34.232768 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f6f8c-44ae-477f-a8a8-c8033de7f7cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:34.233051 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f6f8c-44ae-477f-a8a8-c8033de7f7cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:34.234855 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4b4f6f8c-44ae-477f-a8a8-c8033de7f7cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:34.301363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:34.383869 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f6f8c-44ae-477f-a8a8-c8033de7f7cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:34.386375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f6f8c-44ae-477f-a8a8-c8033de7f7cd service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:34.416837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 662/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:35:36.994364 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b24a9f-806a-450e-804a-d5133311055a None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:36.995002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b24a9f-806a-450e-804a-d5133311055a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:36.995002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b24a9f-806a-450e-804a-d5133311055a None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:36.996254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 650/1312] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:35:37.002062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eeac48-d2e0-4aeb-ba7d-6e4af94020ed None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:37.002540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eeac48-d2e0-4aeb-ba7d-6e4af94020ed None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:37.002636 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eeac48-d2e0-4aeb-ba7d-6e4af94020ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:37.097043 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-62eeac48-d2e0-4aeb-ba7d-6e4af94020ed None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:37.098352 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-62eeac48-d2e0-4aeb-ba7d-6e4af94020ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:37.220266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 663/1313] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:35:37.511205 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b24a9f-806a-450e-804a-d5133311055a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:37.512764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-34b24a9f-806a-450e-804a-d5133311055a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:37.632586 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b24a9f-806a-450e-804a-d5133311055a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:37.719053 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:37.721801 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a1887-4efc-4243-a04e-ff97cb9bb19e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:37.722698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a1887-4efc-4243-a04e-ff97cb9bb19e service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:37.723138 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a1887-4efc-4243-a04e-ff97cb9bb19e service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:37.723577 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a1887-4efc-4243-a04e-ff97cb9bb19e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:37.725266 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c33a1887-4efc-4243-a04e-ff97cb9bb19e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:37.834794 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a1887-4efc-4243-a04e-ff97cb9bb19e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:37.836232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a1887-4efc-4243-a04e-ff97cb9bb19e service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:37.855446 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 651/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:40.374752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde632f7-79b8-402a-9948-778ad5c28515 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:40.375413 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde632f7-79b8-402a-9948-778ad5c28515 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:40.375413 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde632f7-79b8-402a-9948-778ad5c28515 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:40.376691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 664/1315] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:35:40.382664 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5068f0a-13f3-4268-9a35-eee2af0fc4cc None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:40.383202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5068f0a-13f3-4268-9a35-eee2af0fc4cc None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:40.383202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5068f0a-13f3-4268-9a35-eee2af0fc4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:40.453504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c5068f0a-13f3-4268-9a35-eee2af0fc4cc None None] MFA Rules not processed for user `93500b88e77e4d4ba8f347e33daa2e69`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:40.454729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c5068f0a-13f3-4268-9a35-eee2af0fc4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:40.511827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:40.558963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 652/1316] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:35:40 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:40.843437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde632f7-79b8-402a-9948-778ad5c28515 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:40.845051 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-fde632f7-79b8-402a-9948-778ad5c28515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:40.856782 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:40.962244 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde632f7-79b8-402a-9948-778ad5c28515 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:41.044001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4723d26-4298-49b0-8190-9134c36c8dda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:41.044496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4723d26-4298-49b0-8190-9134c36c8dda service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:41.044775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4723d26-4298-49b0-8190-9134c36c8dda service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:41.045042 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4723d26-4298-49b0-8190-9134c36c8dda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:41.046812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c4723d26-4298-49b0-8190-9134c36c8dda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:41.114637 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:41.170078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4723d26-4298-49b0-8190-9134c36c8dda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:41.171525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4723d26-4298-49b0-8190-9134c36c8dda service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:41.191834 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 665/1317] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:35:44.049799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4fdeb-727d-4690-b5e3-c7cc95b7ab94 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:44.050688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4fdeb-727d-4690-b5e3-c7cc95b7ab94 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:44.050688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4fdeb-727d-4690-b5e3-c7cc95b7ab94 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:44.052183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 653/1318] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:35:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:35:44.058281 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ad9e1-d28a-4e4e-ab9f-88c59babdea5 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:44.058805 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ad9e1-d28a-4e4e-ab9f-88c59babdea5 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:44.058805 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ad9e1-d28a-4e4e-ab9f-88c59babdea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:44.131046 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-3d7ad9e1-d28a-4e4e-ab9f-88c59babdea5 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:44.132273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-3d7ad9e1-d28a-4e4e-ab9f-88c59babdea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:44.263627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 666/1319] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:35:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:44.547561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4fdeb-727d-4690-b5e3-c7cc95b7ab94 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:44.549115 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a6c4fdeb-727d-4690-b5e3-c7cc95b7ab94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:44.727399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4fdeb-727d-4690-b5e3-c7cc95b7ab94 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:44.862475 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6833886-bdad-4b56-89c3-40472b94bdd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:44.863495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6833886-bdad-4b56-89c3-40472b94bdd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:44.863977 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6833886-bdad-4b56-89c3-40472b94bdd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:44.864449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6833886-bdad-4b56-89c3-40472b94bdd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:44.867500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c6833886-bdad-4b56-89c3-40472b94bdd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:44.880337 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:45.061017 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6833886-bdad-4b56-89c3-40472b94bdd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:45.062535 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6833886-bdad-4b56-89c3-40472b94bdd6 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:45.083678 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 654/1320] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:35:48.516136 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905121b-ce7d-444d-824c-17e8f421dbbe None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:48.516980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905121b-ce7d-444d-824c-17e8f421dbbe None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:48.516980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905121b-ce7d-444d-824c-17e8f421dbbe None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:48.518087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 667/1321] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:35:48.525306 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73ad8b-5ed5-4e3a-8ad1-fb06bc713add None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:48.526194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73ad8b-5ed5-4e3a-8ad1-fb06bc713add None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:48.526194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73ad8b-5ed5-4e3a-8ad1-fb06bc713add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:48.617026 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-1c73ad8b-5ed5-4e3a-8ad1-fb06bc713add None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:48.618314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1c73ad8b-5ed5-4e3a-8ad1-fb06bc713add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:48.688109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:48.753694 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 655/1322] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:49.076524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905121b-ce7d-444d-824c-17e8f421dbbe None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:49.078434 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f905121b-ce7d-444d-824c-17e8f421dbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:49.105920 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:49.211274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905121b-ce7d-444d-824c-17e8f421dbbe None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:49.300296 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c54ca5-f8ca-4269-bb1b-1accc0a2c74b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:49.300906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c54ca5-f8ca-4269-bb1b-1accc0a2c74b service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:49.301201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c54ca5-f8ca-4269-bb1b-1accc0a2c74b service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:49.301470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c54ca5-f8ca-4269-bb1b-1accc0a2c74b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:49.303297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-42c54ca5-f8ca-4269-bb1b-1accc0a2c74b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:49.382867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:49.425639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c54ca5-f8ca-4269-bb1b-1accc0a2c74b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:49.427140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c54ca5-f8ca-4269-bb1b-1accc0a2c74b service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:49.451323 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 668/1323] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:35:51.709550 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e1268-d7a4-4b4d-bb0b-e5b1bfd4d6fe None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:51.710117 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e1268-d7a4-4b4d-bb0b-e5b1bfd4d6fe None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:51.710117 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e1268-d7a4-4b4d-bb0b-e5b1bfd4d6fe None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:51.711450 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 656/1324] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:35:51.717186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cea900-261f-4640-b162-5fd457a7b21e None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:51.717677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cea900-261f-4640-b162-5fd457a7b21e None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:51.717806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cea900-261f-4640-b162-5fd457a7b21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:51.790995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-c9cea900-261f-4640-b162-5fd457a7b21e None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:51.792220 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c9cea900-261f-4640-b162-5fd457a7b21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:51.897605 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 669/1325] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:52.178609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e1268-d7a4-4b4d-bb0b-e5b1bfd4d6fe None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:52.180285 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-110e1268-d7a4-4b4d-bb0b-e5b1bfd4d6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:52.297523 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e1268-d7a4-4b4d-bb0b-e5b1bfd4d6fe None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:52.384305 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df70f99-a1a1-4c3e-b49f-af9cc6caabba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:52.384929 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df70f99-a1a1-4c3e-b49f-af9cc6caabba service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:52.385236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df70f99-a1a1-4c3e-b49f-af9cc6caabba service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:52.385508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df70f99-a1a1-4c3e-b49f-af9cc6caabba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:52.387169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5df70f99-a1a1-4c3e-b49f-af9cc6caabba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:52.407177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:52.516187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df70f99-a1a1-4c3e-b49f-af9cc6caabba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:52.517678 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df70f99-a1a1-4c3e-b49f-af9cc6caabba service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:52.538855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 657/1326] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:54.791677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1badea-f41e-44a0-84f8-a863b645844f None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:54.792181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1badea-f41e-44a0-84f8-a863b645844f None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:54.792288 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1badea-f41e-44a0-84f8-a863b645844f None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:54.793902 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 670/1327] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:35:54.802113 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a52d5d-5ad3-4dee-be38-790fa4694b9e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:54.802585 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a52d5d-5ad3-4dee-be38-790fa4694b9e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:54.802665 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a52d5d-5ad3-4dee-be38-790fa4694b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:54.886915 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-06a52d5d-5ad3-4dee-be38-790fa4694b9e None None] MFA Rules not processed for user `93500b88e77e4d4ba8f347e33daa2e69`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:54.888279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-06a52d5d-5ad3-4dee-be38-790fa4694b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:54.959879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:54.996752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 658/1328] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:35:54 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:55.282769 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1badea-f41e-44a0-84f8-a863b645844f None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:55.284364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4e1badea-f41e-44a0-84f8-a863b645844f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:55.325759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:55.406942 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1badea-f41e-44a0-84f8-a863b645844f None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:55.504040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa7c55-ed11-4eff-90d8-7c6067928af4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:55.504706 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa7c55-ed11-4eff-90d8-7c6067928af4 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:55.505114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa7c55-ed11-4eff-90d8-7c6067928af4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:55.505385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa7c55-ed11-4eff-90d8-7c6067928af4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:55.507615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-54aa7c55-ed11-4eff-90d8-7c6067928af4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:55.603088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:55.641893 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa7c55-ed11-4eff-90d8-7c6067928af4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:55.643337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa7c55-ed11-4eff-90d8-7c6067928af4 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:55.664670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 671/1329] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:55 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:35:58.111136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef423f-d417-4fdf-a6ee-36044fb27be8 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:58.111796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef423f-d417-4fdf-a6ee-36044fb27be8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:58.111796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef423f-d417-4fdf-a6ee-36044fb27be8 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:58.113061 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 659/1330] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:35:58.118459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001d3b4-f357-4536-bd8d-b1558c506c8c None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:58.118916 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001d3b4-f357-4536-bd8d-b1558c506c8c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:58.119035 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001d3b4-f357-4536-bd8d-b1558c506c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:58.205861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-1001d3b4-f357-4536-bd8d-b1558c506c8c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:58.207612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-1001d3b4-f357-4536-bd8d-b1558c506c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:58.326185 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 672/1331] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:35:58.635645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb66a5-37a2-4d80-b5cc-e261f0e98f82 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:58.635936 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb66a5-37a2-4d80-b5cc-e261f0e98f82 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:58.636201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb66a5-37a2-4d80-b5cc-e261f0e98f82 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:58.637872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 660/1332] 10.222.0.57 () {54 vars in 964 bytes} [Mon Sep 26 05:35:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:35:58.644255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21f809-d66c-42b1-92f4-0bbd4e63cd36 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:58.644493 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21f809-d66c-42b1-92f4-0bbd4e63cd36 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:58.644802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21f809-d66c-42b1-92f4-0bbd4e63cd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:58.698741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:58.733855 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-ee21f809-d66c-42b1-92f4-0bbd4e63cd36 None None] MFA Rules not processed for user `6bf0f594ce7646b7b6ddb8a8553509d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:35:58.735102 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ee21f809-d66c-42b1-92f4-0bbd4e63cd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:58.821956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 673/1333] 10.222.0.57 () {58 vars in 1061 bytes} [Mon Sep 26 05:35:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:35:58.831438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb66a5-37a2-4d80-b5cc-e261f0e98f82 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:35:58.833015 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-51eb66a5-37a2-4d80-b5cc-e261f0e98f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:58.951230 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb66a5-37a2-4d80-b5cc-e261f0e98f82 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:35:59.032625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22753d81-51be-4240-bbbe-ca54c6f1520a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:35:59.033344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22753d81-51be-4240-bbbe-ca54c6f1520a service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:35:59.033619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22753d81-51be-4240-bbbe-ca54c6f1520a service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:35:59.033891 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22753d81-51be-4240-bbbe-ca54c6f1520a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:35:59.035508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-22753d81-51be-4240-bbbe-ca54c6f1520a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:35:59.064288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:35:59.158572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22753d81-51be-4240-bbbe-ca54c6f1520a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:35:59.160027 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22753d81-51be-4240-bbbe-ca54c6f1520a service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:35:59.179353 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 661/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:01.676920 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc1625-cf2b-44be-8ae0-aced16b16b24 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:01.677536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc1625-cf2b-44be-8ae0-aced16b16b24 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:01.677536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc1625-cf2b-44be-8ae0-aced16b16b24 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:01.678891 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 674/1335] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:36:01.684694 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad678d-543b-4d1d-8926-39501cd8c2ca None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:01.685115 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad678d-543b-4d1d-8926-39501cd8c2ca None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:01.685271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad678d-543b-4d1d-8926-39501cd8c2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:01.763234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-9aad678d-543b-4d1d-8926-39501cd8c2ca None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:01.764468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9aad678d-543b-4d1d-8926-39501cd8c2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:01.843830 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:01.867194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 662/1336] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:02.152280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc1625-cf2b-44be-8ae0-aced16b16b24 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:02.153893 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-fcfc1625-cf2b-44be-8ae0-aced16b16b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:02.264295 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc1625-cf2b-44be-8ae0-aced16b16b24 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:02.293733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:02.347651 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53ce1c-e249-4200-bb3e-94df40def7d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:02.348265 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53ce1c-e249-4200-bb3e-94df40def7d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:02.348540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53ce1c-e249-4200-bb3e-94df40def7d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:02.348808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53ce1c-e249-4200-bb3e-94df40def7d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:02.350488 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ff53ce1c-e249-4200-bb3e-94df40def7d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:02.467599 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53ce1c-e249-4200-bb3e-94df40def7d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:02.468967 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53ce1c-e249-4200-bb3e-94df40def7d3 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:02.488445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 675/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:05.066938 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac3eb1-bfac-4553-a8dd-c91cefba22b6 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:05.067559 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac3eb1-bfac-4553-a8dd-c91cefba22b6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:05.067559 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac3eb1-bfac-4553-a8dd-c91cefba22b6 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:05.068880 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 663/1338] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:36:05.076799 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898047f0-ef1a-40f7-9a5e-8401fe3e8b27 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:05.077529 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898047f0-ef1a-40f7-9a5e-8401fe3e8b27 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:05.077529 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898047f0-ef1a-40f7-9a5e-8401fe3e8b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:05.149312 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-898047f0-ef1a-40f7-9a5e-8401fe3e8b27 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:05.151112 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-898047f0-ef1a-40f7-9a5e-8401fe3e8b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:05.160302 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:05.251581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 676/1339] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:36:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:05.529464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac3eb1-bfac-4553-a8dd-c91cefba22b6 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:05.531079 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-89ac3eb1-bfac-4553-a8dd-c91cefba22b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:05.655122 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac3eb1-bfac-4553-a8dd-c91cefba22b6 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:05.737121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05006cc-6abf-4020-b999-29878a38bec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:05.737782 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05006cc-6abf-4020-b999-29878a38bec0 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:05.738041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05006cc-6abf-4020-b999-29878a38bec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:05.738304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05006cc-6abf-4020-b999-29878a38bec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:05.739934 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a05006cc-6abf-4020-b999-29878a38bec0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:05.779309 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:05.863090 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05006cc-6abf-4020-b999-29878a38bec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:05.864524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05006cc-6abf-4020-b999-29878a38bec0 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:05.883758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 664/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:10.090118 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63501120-04d2-492c-be32-2eea11ac5e7e None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:10.090773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63501120-04d2-492c-be32-2eea11ac5e7e None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:10.090773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63501120-04d2-492c-be32-2eea11ac5e7e None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:10.093470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 677/1341] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:36:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:36:10.097938 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff46323-3ead-40de-b971-378531c9f79c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:10.098431 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff46323-3ead-40de-b971-378531c9f79c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:10.098520 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff46323-3ead-40de-b971-378531c9f79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:10.169941 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-bff46323-3ead-40de-b971-378531c9f79c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:10.171159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bff46323-3ead-40de-b971-378531c9f79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:10.277858 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:10.290610 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 665/1342] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:10.575772 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63501120-04d2-492c-be32-2eea11ac5e7e None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:10.577883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-63501120-04d2-492c-be32-2eea11ac5e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:10.705725 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63501120-04d2-492c-be32-2eea11ac5e7e None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:10.749862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:10.788860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab86f6e-bbb5-4a39-b8e3-07565bba3f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:10.789503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab86f6e-bbb5-4a39-b8e3-07565bba3f1c service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:10.789779 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab86f6e-bbb5-4a39-b8e3-07565bba3f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:10.790040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab86f6e-bbb5-4a39-b8e3-07565bba3f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:10.791754 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-aab86f6e-bbb5-4a39-b8e3-07565bba3f1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:10.902351 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab86f6e-bbb5-4a39-b8e3-07565bba3f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:10.903661 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab86f6e-bbb5-4a39-b8e3-07565bba3f1c service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:10.925922 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 678/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:13.809158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525da99-972f-4147-9318-6e7ece9aa282 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:13.809741 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525da99-972f-4147-9318-6e7ece9aa282 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:13.809741 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525da99-972f-4147-9318-6e7ece9aa282 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:13.811079 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 666/1344] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:36:13.816616 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c25f3-9dba-4fed-82ca-40026cc07794 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:13.817108 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c25f3-9dba-4fed-82ca-40026cc07794 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:13.817245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c25f3-9dba-4fed-82ca-40026cc07794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:13.886658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-404c25f3-9dba-4fed-82ca-40026cc07794 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:13.887860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-404c25f3-9dba-4fed-82ca-40026cc07794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:13.907137 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:14.000905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 679/1345] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:36:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:14.317298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525da99-972f-4147-9318-6e7ece9aa282 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:14.319034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7525da99-972f-4147-9318-6e7ece9aa282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:14.434929 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525da99-972f-4147-9318-6e7ece9aa282 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:14.515216 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433b5df-f3a5-4836-930c-ff26c7ec7aaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:14.515738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433b5df-f3a5-4836-930c-ff26c7ec7aaf service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:14.516016 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433b5df-f3a5-4836-930c-ff26c7ec7aaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:14.516286 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433b5df-f3a5-4836-930c-ff26c7ec7aaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:14.517939 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d433b5df-f3a5-4836-930c-ff26c7ec7aaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:14.562208 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:14.632743 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d433b5df-f3a5-4836-930c-ff26c7ec7aaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:14.634221 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d433b5df-f3a5-4836-930c-ff26c7ec7aaf service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:14.653422 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 667/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:18.884468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88fa29-78d5-4896-a1a9-2565740f1901 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:18.884882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88fa29-78d5-4896-a1a9-2565740f1901 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:18.885095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88fa29-78d5-4896-a1a9-2565740f1901 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:18.886521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 680/1347] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:36:18.892135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4985a-361b-4454-b639-b67dd5ece0af None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:18.892636 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4985a-361b-4454-b639-b67dd5ece0af None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:18.892719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4985a-361b-4454-b639-b67dd5ece0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:18.969504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d3a4985a-361b-4454-b639-b67dd5ece0af None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:18.970685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d3a4985a-361b-4454-b639-b67dd5ece0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:19.085697 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:19.088965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 668/1348] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:19.372689 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88fa29-78d5-4896-a1a9-2565740f1901 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:19.374310 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ce88fa29-78d5-4896-a1a9-2565740f1901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:19.487292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88fa29-78d5-4896-a1a9-2565740f1901 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:19.539684 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:19.566280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85936a9b-1cfc-4a54-a5e8-680d54df40f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:19.566892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85936a9b-1cfc-4a54-a5e8-680d54df40f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:19.567166 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85936a9b-1cfc-4a54-a5e8-680d54df40f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:19.567431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85936a9b-1cfc-4a54-a5e8-680d54df40f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:19.569044 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-85936a9b-1cfc-4a54-a5e8-680d54df40f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:19.681711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85936a9b-1cfc-4a54-a5e8-680d54df40f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:19.683195 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85936a9b-1cfc-4a54-a5e8-680d54df40f8 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:19.703473 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 681/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:24.982307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061560a-9c99-4b64-8126-af1ac7c5eab9 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:24.982755 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061560a-9c99-4b64-8126-af1ac7c5eab9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:24.983223 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061560a-9c99-4b64-8126-af1ac7c5eab9 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:24.986400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 669/1350] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 26 05:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:36:24.999494 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dba378-43dd-4854-a8c5-b8c104ed16b0 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:25.000009 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dba378-43dd-4854-a8c5-b8c104ed16b0 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:25.000576 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dba378-43dd-4854-a8c5-b8c104ed16b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:25.126982 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-96dba378-43dd-4854-a8c5-b8c104ed16b0 None None] MFA Rules not processed for user `67fcd65eeb994b2381b83480e030f2ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:25.128227 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-96dba378-43dd-4854-a8c5-b8c104ed16b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:25.161708 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:25.304084 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 682/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 26 05:36:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:29.809850 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a65e70-7ce2-401f-9e72-6255bef71389 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:29.811157 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a65e70-7ce2-401f-9e72-6255bef71389 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:29.811844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a65e70-7ce2-401f-9e72-6255bef71389 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:29.815116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 670/1352] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 26 05:36:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:36:29.824917 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0bc47-d9e0-41c6-be8c-c6163353948d None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:29.826518 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0bc47-d9e0-41c6-be8c-c6163353948d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:29.827020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0bc47-d9e0-41c6-be8c-c6163353948d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:29.950987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-2dc0bc47-d9e0-41c6-be8c-c6163353948d None None] MFA Rules not processed for user `67fcd65eeb994b2381b83480e030f2ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:29.952124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2dc0bc47-d9e0-41c6-be8c-c6163353948d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:30.046654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:30.090328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 683/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 26 05:36:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:31.288774 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279176e8-1e87-4aea-9404-0fd62a22dd7b None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:31.289369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279176e8-1e87-4aea-9404-0fd62a22dd7b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:31.289369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279176e8-1e87-4aea-9404-0fd62a22dd7b None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:31.290668 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 671/1354] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 26 05:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:36:31.296931 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa33c12-e232-44be-aaae-aaf8d6a52549 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:31.297684 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa33c12-e232-44be-aaae-aaf8d6a52549 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:31.297684 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa33c12-e232-44be-aaae-aaf8d6a52549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:31.396355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-6aa33c12-e232-44be-aaae-aaf8d6a52549 None None] MFA Rules not processed for user `67fcd65eeb994b2381b83480e030f2ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:31.397602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6aa33c12-e232-44be-aaae-aaf8d6a52549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:31.518999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 684/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 26 05:36:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:35.329119 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4238a6-eae0-4c58-8761-c79d4ef246ab None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:35.330028 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4238a6-eae0-4c58-8761-c79d4ef246ab None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:35.330450 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4238a6-eae0-4c58-8761-c79d4ef246ab None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:35.333344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 672/1356] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:36:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:36:35.338982 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5eee5-6c71-48c9-b1fe-e0b77a727eb6 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:35.340757 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5eee5-6c71-48c9-b1fe-e0b77a727eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:35.341200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5eee5-6c71-48c9-b1fe-e0b77a727eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:35.400695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:35.447765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-f6d5eee5-6c71-48c9-b1fe-e0b77a727eb6 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:35.449452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f6d5eee5-6c71-48c9-b1fe-e0b77a727eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:35.592362 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 685/1357] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:35.898339 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b768f-258b-423c-b014-cc0e2ceba9c2 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:35.898622 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b768f-258b-423c-b014-cc0e2ceba9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:35.898896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b768f-258b-423c-b014-cc0e2ceba9c2 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:35.900301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 673/1358] 10.222.0.57 () {54 vars in 961 bytes} [Mon Sep 26 05:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:36:35.910357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8d2d4-799f-4a09-b282-f0df6d407d2d None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:35.910755 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8d2d4-799f-4a09-b282-f0df6d407d2d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:35.911138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8d2d4-799f-4a09-b282-f0df6d407d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:36.008007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-7cb8d2d4-799f-4a09-b282-f0df6d407d2d None None] MFA Rules not processed for user `92556048cbd04c409c3f4b6fd645627f`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:36.009270 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7cb8d2d4-799f-4a09-b282-f0df6d407d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:36.076259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:36.175008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 686/1359] 10.222.0.57 () {58 vars in 1058 bytes} [Mon Sep 26 05:36:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:36.182032 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b768f-258b-423c-b014-cc0e2ceba9c2 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:36.183554 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0a4b768f-258b-423c-b014-cc0e2ceba9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:36.359979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b768f-258b-423c-b014-cc0e2ceba9c2 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:36.441524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:36.486701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f50ac3-7b45-4e29-a5d6-b36ab483ce90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:36.487506 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f50ac3-7b45-4e29-a5d6-b36ab483ce90 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:36.488032 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f50ac3-7b45-4e29-a5d6-b36ab483ce90 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:36.488477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f50ac3-7b45-4e29-a5d6-b36ab483ce90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:36.490270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a7f50ac3-7b45-4e29-a5d6-b36ab483ce90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:36.608477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f50ac3-7b45-4e29-a5d6-b36ab483ce90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:36.610354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f50ac3-7b45-4e29-a5d6-b36ab483ce90 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:36.632670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 674/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:39.111255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1b3c2-a0e2-47d8-b9b0-4e5351ae0827 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:39.111255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1b3c2-a0e2-47d8-b9b0-4e5351ae0827 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:39.112559 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1b3c2-a0e2-47d8-b9b0-4e5351ae0827 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:39.119737 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4704bde-bffb-49dc-98e8-572c6097a115 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:39.119737 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4704bde-bffb-49dc-98e8-572c6097a115 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:39.119737 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4704bde-bffb-49dc-98e8-572c6097a115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:39.125167 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 687/1361] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:36:39 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:36:39.205315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a4704bde-bffb-49dc-98e8-572c6097a115 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:39.207379 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a4704bde-bffb-49dc-98e8-572c6097a115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:39.267294 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:39.349993 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 675/1362] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:39.859154 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1b3c2-a0e2-47d8-b9b0-4e5351ae0827 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:39.860786 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-08b1b3c2-a0e2-47d8-b9b0-4e5351ae0827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:40.126483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1b3c2-a0e2-47d8-b9b0-4e5351ae0827 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:40.139075 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:40.325181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b31ac-8e16-4ff6-8000-7356b362daa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:40.325181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b31ac-8e16-4ff6-8000-7356b362daa0 service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:40.325181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b31ac-8e16-4ff6-8000-7356b362daa0 service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:40.325181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b31ac-8e16-4ff6-8000-7356b362daa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:40.326002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-8e7b31ac-8e16-4ff6-8000-7356b362daa0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:40.457491 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7b31ac-8e16-4ff6-8000-7356b362daa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:40.459142 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7b31ac-8e16-4ff6-8000-7356b362daa0 service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:40.483858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 688/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:36:43.517604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e7e61-1fb2-44d5-867a-ece4e6d2fb7e None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:43.518512 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e7e61-1fb2-44d5-867a-ece4e6d2fb7e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:43.518512 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e7e61-1fb2-44d5-867a-ece4e6d2fb7e None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:43.521196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 676/1364] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:36:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:36:43.531004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741c29c-c151-4bb3-af5f-1a70f2c21ebb None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:43.531862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741c29c-c151-4bb3-af5f-1a70f2c21ebb None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:43.532041 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741c29c-c151-4bb3-af5f-1a70f2c21ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:43.566110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:43.675465 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-a741c29c-c151-4bb3-af5f-1a70f2c21ebb None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:43.677534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-a741c29c-c151-4bb3-af5f-1a70f2c21ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:43.833324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 689/1365] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:44.359167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362f344-acde-4180-88c7-06eaced4ee2b None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:44.359712 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362f344-acde-4180-88c7-06eaced4ee2b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:44.360116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362f344-acde-4180-88c7-06eaced4ee2b None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:44.361904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 677/1366] 10.222.0.57 () {54 vars in 961 bytes} [Mon Sep 26 05:36:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:36:44.371792 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d725e-0837-468d-879f-0648f0db0933 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:44.372251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d725e-0837-468d-879f-0648f0db0933 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:44.372685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d725e-0837-468d-879f-0648f0db0933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:44.481749 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-aa5d725e-0837-468d-879f-0648f0db0933 None None] MFA Rules not processed for user `92556048cbd04c409c3f4b6fd645627f`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:44.483306 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-aa5d725e-0837-468d-879f-0648f0db0933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:44.534291 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:44.651770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 690/1367] 10.222.0.57 () {58 vars in 1058 bytes} [Mon Sep 26 05:36:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:44.662325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362f344-acde-4180-88c7-06eaced4ee2b None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:44.664289 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7362f344-acde-4180-88c7-06eaced4ee2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:44.851352 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362f344-acde-4180-88c7-06eaced4ee2b None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:44.883333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:44.907860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7ed20-9b71-4141-bf6a-739cea99dfae None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:44.908397 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7ed20-9b71-4141-bf6a-739cea99dfae None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:44.908993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7ed20-9b71-4141-bf6a-739cea99dfae None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:44.911857 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 691/1368] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:36:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:36:44.918164 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0925a-1a3d-49a2-b3c0-80ffc9ea30d5 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:44.918445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0925a-1a3d-49a2-b3c0-80ffc9ea30d5 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:44.919159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0925a-1a3d-49a2-b3c0-80ffc9ea30d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:45.004017 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb48ed-1326-4285-81b5-e4bb4aad3608 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:45.004628 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb48ed-1326-4285-81b5-e4bb4aad3608 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:45.004902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb48ed-1326-4285-81b5-e4bb4aad3608 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:45.005207 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb48ed-1326-4285-81b5-e4bb4aad3608 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:45.006836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c5cb48ed-1326-4285-81b5-e4bb4aad3608 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:45.025339 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-12f0925a-1a3d-49a2-b3c0-80ffc9ea30d5 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:45.027553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-12f0925a-1a3d-49a2-b3c0-80ffc9ea30d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:45.131910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb48ed-1326-4285-81b5-e4bb4aad3608 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:45.135511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb48ed-1326-4285-81b5-e4bb4aad3608 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:45.168752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:45.174042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 678/1369] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:45.178763 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 692/1370] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:36:45.486427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405a12d-3f25-4888-9ea4-9fa9fd657007 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:45.486706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405a12d-3f25-4888-9ea4-9fa9fd657007 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:45.486976 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405a12d-3f25-4888-9ea4-9fa9fd657007 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:45.488448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 679/1371] 10.222.0.57 () {54 vars in 963 bytes} [Mon Sep 26 05:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:36:45.495902 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221763e4-c632-4fca-927f-ba5a8d7037b3 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:45.495902 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221763e4-c632-4fca-927f-ba5a8d7037b3 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:45.495902 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221763e4-c632-4fca-927f-ba5a8d7037b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:45.592048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-221763e4-c632-4fca-927f-ba5a8d7037b3 None None] MFA Rules not processed for user `46edd6aba77c4404977b1c39e8442b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:45.593419 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-221763e4-c632-4fca-927f-ba5a8d7037b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:45.692360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 693/1372] 10.222.0.57 () {58 vars in 1060 bytes} [Mon Sep 26 05:36:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:45.700973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405a12d-3f25-4888-9ea4-9fa9fd657007 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:45.702920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3405a12d-3f25-4888-9ea4-9fa9fd657007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:45.741934 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:45.867292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405a12d-3f25-4888-9ea4-9fa9fd657007 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:45.973596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cbadef-d189-42ff-931f-3aba8e1505d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:45.974224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbadef-d189-42ff-931f-3aba8e1505d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:45.974495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbadef-d189-42ff-931f-3aba8e1505d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:45.974825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbadef-d189-42ff-931f-3aba8e1505d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:45.976936 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d6cbadef-d189-42ff-931f-3aba8e1505d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:46.090781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:46.142433 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cbadef-d189-42ff-931f-3aba8e1505d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:46.144391 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cbadef-d189-42ff-931f-3aba8e1505d8 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:46.171174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 680/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:47.516838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c77f72-5386-4819-8ba1-e95551251e06 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:47.517642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c77f72-5386-4819-8ba1-e95551251e06 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:47.517642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c77f72-5386-4819-8ba1-e95551251e06 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:47.518928 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 694/1374] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:36:47.524540 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70e349-fbf4-480f-a741-4c5e9d50af41 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:47.525169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70e349-fbf4-480f-a741-4c5e9d50af41 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:47.525169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70e349-fbf4-480f-a741-4c5e9d50af41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:47.596411 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-eb70e349-fbf4-480f-a741-4c5e9d50af41 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:47.597686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-eb70e349-fbf4-480f-a741-4c5e9d50af41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:47.697629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 681/1375] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:36:48.002243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c77f72-5386-4819-8ba1-e95551251e06 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:48.003831 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-a1c77f72-5386-4819-8ba1-e95551251e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:48.101644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:48.168375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c77f72-5386-4819-8ba1-e95551251e06 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:48.310392 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0436f64-1d49-4d1f-a682-22ad9d9439bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:48.311221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0436f64-1d49-4d1f-a682-22ad9d9439bc service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:48.311521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0436f64-1d49-4d1f-a682-22ad9d9439bc service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:48.311914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0436f64-1d49-4d1f-a682-22ad9d9439bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:48.313576 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-b0436f64-1d49-4d1f-a682-22ad9d9439bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:48.378492 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:48.428617 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0436f64-1d49-4d1f-a682-22ad9d9439bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:48.430313 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0436f64-1d49-4d1f-a682-22ad9d9439bc service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:48.451341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 695/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:50.795691 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3d086-71e1-46d0-a654-47c23e357a0a None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:50.796257 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3d086-71e1-46d0-a654-47c23e357a0a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:50.796594 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3d086-71e1-46d0-a654-47c23e357a0a None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:50.802157 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 682/1377] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:36:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:36:50.808695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c19b42-f49c-4427-839b-c78dfbd7bca1 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:50.809114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c19b42-f49c-4427-839b-c78dfbd7bca1 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:50.809259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c19b42-f49c-4427-839b-c78dfbd7bca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:50.923896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d1c19b42-f49c-4427-839b-c78dfbd7bca1 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:50.925723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d1c19b42-f49c-4427-839b-c78dfbd7bca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:51.059378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 696/1378] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:36:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:51.343750 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3d086-71e1-46d0-a654-47c23e357a0a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:51.345796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c9e3d086-71e1-46d0-a654-47c23e357a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:51.386829 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:51.520910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3d086-71e1-46d0-a654-47c23e357a0a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:51.667423 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:51.673349 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be362a4c-2d4d-40c9-9d7a-f244948b3568 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:51.674136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be362a4c-2d4d-40c9-9d7a-f244948b3568 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:51.674416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be362a4c-2d4d-40c9-9d7a-f244948b3568 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:51.674684 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be362a4c-2d4d-40c9-9d7a-f244948b3568 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:51.677281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-be362a4c-2d4d-40c9-9d7a-f244948b3568 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:51.810206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be362a4c-2d4d-40c9-9d7a-f244948b3568 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:51.815006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be362a4c-2d4d-40c9-9d7a-f244948b3568 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:51.850714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 683/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:54.271645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f5325-4b6d-40c1-8a9c-4492029ea196 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:54.272558 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f5325-4b6d-40c1-8a9c-4492029ea196 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:54.273000 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f5325-4b6d-40c1-8a9c-4492029ea196 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:54.274472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 697/1380] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:36:54.284650 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76cb17-449a-43b4-a355-891f64b39e24 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:54.285562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76cb17-449a-43b4-a355-891f64b39e24 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:54.285836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76cb17-449a-43b4-a355-891f64b39e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:54.362713 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-0d76cb17-449a-43b4-a355-891f64b39e24 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:54.364109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0d76cb17-449a-43b4-a355-891f64b39e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:54.460204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:54.504600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 684/1381] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:36:54.827829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f5325-4b6d-40c1-8a9c-4492029ea196 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:54.827829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-851f5325-4b6d-40c1-8a9c-4492029ea196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:54.852808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:55.001626 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f5325-4b6d-40c1-8a9c-4492029ea196 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:55.160092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:55.180499 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede01c1-4ec7-4e6e-892c-50464cbdc884 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:55.181234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede01c1-4ec7-4e6e-892c-50464cbdc884 service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:55.181507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede01c1-4ec7-4e6e-892c-50464cbdc884 service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:55.181793 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede01c1-4ec7-4e6e-892c-50464cbdc884 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:55.183626 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7ede01c1-4ec7-4e6e-892c-50464cbdc884 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:55.299504 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede01c1-4ec7-4e6e-892c-50464cbdc884 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:55.301326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede01c1-4ec7-4e6e-892c-50464cbdc884 service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:55.321802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 698/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:36:57.750124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267a494-c8f9-4bfb-9b8e-646226d4c2f3 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:57.750725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267a494-c8f9-4bfb-9b8e-646226d4c2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:57.750725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267a494-c8f9-4bfb-9b8e-646226d4c2f3 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:57.752225 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 685/1383] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:36:57.758776 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e2f90-38dd-47d9-81b4-c01e3bedc271 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:57.759851 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e2f90-38dd-47d9-81b4-c01e3bedc271 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:57.759851 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e2f90-38dd-47d9-81b4-c01e3bedc271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:57.841966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d81e2f90-38dd-47d9-81b4-c01e3bedc271 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:36:57.843309 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d81e2f90-38dd-47d9-81b4-c01e3bedc271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:57.918639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:57.978236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 699/1384] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:36:58.369246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267a494-c8f9-4bfb-9b8e-646226d4c2f3 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:36:58.371034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5267a494-c8f9-4bfb-9b8e-646226d4c2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:58.569893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267a494-c8f9-4bfb-9b8e-646226d4c2f3 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:36:58.631439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:36:58.736883 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7e1eb-cab3-4ec7-9abb-642b42752eea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:36:58.737950 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7e1eb-cab3-4ec7-9abb-642b42752eea service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:36:58.738459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7e1eb-cab3-4ec7-9abb-642b42752eea service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:36:58.738982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7e1eb-cab3-4ec7-9abb-642b42752eea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:36:58.742094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-88d7e1eb-cab3-4ec7-9abb-642b42752eea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:36:58.870458 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7e1eb-cab3-4ec7-9abb-642b42752eea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:36:58.872040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7e1eb-cab3-4ec7-9abb-642b42752eea service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:36:58.892660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 686/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:36:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:01.205238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc802b-c60b-4703-9b2b-e46c678a2bcf None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:01.205829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc802b-c60b-4703-9b2b-e46c678a2bcf None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:01.205829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc802b-c60b-4703-9b2b-e46c678a2bcf None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:01.207585 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 700/1386] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:37:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:01.213827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57f06f-86ff-492c-989b-2dab3dae2ee8 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:01.214752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57f06f-86ff-492c-989b-2dab3dae2ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:01.215170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57f06f-86ff-492c-989b-2dab3dae2ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:01.289738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8e57f06f-86ff-492c-989b-2dab3dae2ee8 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:01.290979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8e57f06f-86ff-492c-989b-2dab3dae2ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:01.299859 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:01.420223 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 687/1387] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:01.710759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc802b-c60b-4703-9b2b-e46c678a2bcf None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:01.712314 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-8dbc802b-c60b-4703-9b2b-e46c678a2bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:01.913000 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc802b-c60b-4703-9b2b-e46c678a2bcf None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:01.966574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:02.036646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421b537-3a71-4d28-b4ca-4113f231c691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:02.037296 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421b537-3a71-4d28-b4ca-4113f231c691 service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.037570 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421b537-3a71-4d28-b4ca-4113f231c691 service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.037831 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421b537-3a71-4d28-b4ca-4113f231c691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.039474 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-0421b537-3a71-4d28-b4ca-4113f231c691 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:02.045059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c4bb5-59c1-464b-9474-9cfe4ba0f764 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.045358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c4bb5-59c1-464b-9474-9cfe4ba0f764 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.045634 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c4bb5-59c1-464b-9474-9cfe4ba0f764 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.047017 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 688/1388] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:37:02.052600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372a4fc-342c-45cb-973c-4b1d5d41b0e3 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.052885 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372a4fc-342c-45cb-973c-4b1d5d41b0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.053159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372a4fc-342c-45cb-973c-4b1d5d41b0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.126511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-6372a4fc-342c-45cb-973c-4b1d5d41b0e3 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:02.127803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6372a4fc-342c-45cb-973c-4b1d5d41b0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:02.149585 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421b537-3a71-4d28-b4ca-4113f231c691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:02.151785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421b537-3a71-4d28-b4ca-4113f231c691 service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:02.172315 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 701/1389] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:02.222142 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:02.235119 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 689/1390] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:37:02.538650 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c80dd-8014-409a-83b9-6cdd569a63e3 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.539247 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c80dd-8014-409a-83b9-6cdd569a63e3 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.539460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c80dd-8014-409a-83b9-6cdd569a63e3 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.541244 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 702/1391] 10.222.0.57 () {54 vars in 962 bytes} [Mon Sep 26 05:37:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:02.547610 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595cb04-62e3-4387-ab02-1e21349472d5 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.547909 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595cb04-62e3-4387-ab02-1e21349472d5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.548146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595cb04-62e3-4387-ab02-1e21349472d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.650370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-6595cb04-62e3-4387-ab02-1e21349472d5 None None] MFA Rules not processed for user `90c0fabecff2457e931c8c1c36681197`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:02.651635 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6595cb04-62e3-4387-ab02-1e21349472d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:02.745903 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 690/1392] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Sep 26 05:37:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:02.755293 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c80dd-8014-409a-83b9-6cdd569a63e3 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:02.756806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f62c80dd-8014-409a-83b9-6cdd569a63e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:02.820390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:02.877286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c80dd-8014-409a-83b9-6cdd569a63e3 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:02.958322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c631ce-aabc-443b-897d-87d2cc4e2430 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c0fabecff2457e931c8c1c36681197', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:02.958934 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c631ce-aabc-443b-897d-87d2cc4e2430 service glance] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:02.959213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c631ce-aabc-443b-897d-87d2cc4e2430 service glance] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:02.959481 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c631ce-aabc-443b-897d-87d2cc4e2430 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:02.961088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-57c631ce-aabc-443b-897d-87d2cc4e2430 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:03.076188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:03.090496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c631ce-aabc-443b-897d-87d2cc4e2430 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:03.091959 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c631ce-aabc-443b-897d-87d2cc4e2430 service glance] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:03.112843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 703/1393] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:03.928005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6bc0f-af3a-44c0-89a3-12a3f022c5d7 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:03.928553 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6bc0f-af3a-44c0-89a3-12a3f022c5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:03.928553 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6bc0f-af3a-44c0-89a3-12a3f022c5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.000766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-6ed6bc0f-af3a-44c0-89a3-12a3f022c5d7 None None] MFA Rules not processed for user `0216bf67389e4627bf953b7dcc97a88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:04.002010 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6ed6bc0f-af3a-44c0-89a3-12a3f022c5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:04.038442 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:04.096446 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 691/1394] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 26 05:37:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:04.139232 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb1b63-e46c-4e04-ab1c-fac38348a5b6 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.139742 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb1b63-e46c-4e04-ab1c-fac38348a5b6 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.139907 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb1b63-e46c-4e04-ab1c-fac38348a5b6 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.141405 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 704/1395] 10.222.0.57 () {54 vars in 943 bytes} [Mon Sep 26 05:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:04.147803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d59bac-9bab-43b0-8d39-f4e5a8707fd5 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.148198 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d59bac-9bab-43b0-8d39-f4e5a8707fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.148471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d59bac-9bab-43b0-8d39-f4e5a8707fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.247300 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c7d59bac-9bab-43b0-8d39-f4e5a8707fd5 None None] MFA Rules not processed for user `00f258aba84d4dd59ebe8de973ac65c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:04.248519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c7d59bac-9bab-43b0-8d39-f4e5a8707fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:04.391822 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 692/1396] 10.222.0.57 () {58 vars in 1040 bytes} [Mon Sep 26 05:37:04 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:04.401849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb1b63-e46c-4e04-ab1c-fac38348a5b6 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:04.403451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-91cb1b63-e46c-4e04-ab1c-fac38348a5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:04.488815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca669c5-795d-4fb7-b366-845fbfa70dcb None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.489099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca669c5-795d-4fb7-b366-845fbfa70dcb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.489716 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca669c5-795d-4fb7-b366-845fbfa70dcb None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.491136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 693/1397] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:37:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:04.496461 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579af1b4-dd3c-44aa-8eec-586bbe9dcd64 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.496739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579af1b4-dd3c-44aa-8eec-586bbe9dcd64 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.497123 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579af1b4-dd3c-44aa-8eec-586bbe9dcd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.507245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:04.550750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb1b63-e46c-4e04-ab1c-fac38348a5b6 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:04.571566 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-579af1b4-dd3c-44aa-8eec-586bbe9dcd64 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:04.572798 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-579af1b4-dd3c-44aa-8eec-586bbe9dcd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:04.634139 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:04.685260 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b711f5-503f-429c-91c4-1fc452d4c487 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f258aba84d4dd59ebe8de973ac65c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:04.685763 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b711f5-503f-429c-91c4-1fc452d4c487 service swift] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:04.686036 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b711f5-503f-429c-91c4-1fc452d4c487 service swift] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:04.686314 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b711f5-503f-429c-91c4-1fc452d4c487 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:04.688081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-01b711f5-503f-429c-91c4-1fc452d4c487 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:04.693072 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 694/1398] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:04.803327 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b711f5-503f-429c-91c4-1fc452d4c487 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:04.806271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b711f5-503f-429c-91c4-1fc452d4c487 service swift] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:04.822946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 705/1399] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 26 05:37:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:37:04.986902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579af1b4-dd3c-44aa-8eec-586bbe9dcd64 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:04.988854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-579af1b4-dd3c-44aa-8eec-586bbe9dcd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:05.151510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:05.192617 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579af1b4-dd3c-44aa-8eec-586bbe9dcd64 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:05.379376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952003f-287e-473d-8bad-592d5419b825 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:05.380440 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952003f-287e-473d-8bad-592d5419b825 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:05.380954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952003f-287e-473d-8bad-592d5419b825 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:05.381479 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952003f-287e-473d-8bad-592d5419b825 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:05.384925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4952003f-287e-473d-8bad-592d5419b825 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:05.526930 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:05.541943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952003f-287e-473d-8bad-592d5419b825 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:05.544060 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952003f-287e-473d-8bad-592d5419b825 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:05.578168 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 695/1400] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:37:07.964987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7ccc2-6a63-4bbc-9f16-ee1ef5f791a4 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:07.965748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7ccc2-6a63-4bbc-9f16-ee1ef5f791a4 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:07.965748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7ccc2-6a63-4bbc-9f16-ee1ef5f791a4 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:07.966956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 706/1401] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:07.974192 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa136029-2f80-4a9c-b6e5-2a9a2751b508 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:07.982125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa136029-2f80-4a9c-b6e5-2a9a2751b508 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:07.982622 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa136029-2f80-4a9c-b6e5-2a9a2751b508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:08.122236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-aa136029-2f80-4a9c-b6e5-2a9a2751b508 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:08.124220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-aa136029-2f80-4a9c-b6e5-2a9a2751b508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:08.291357 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 696/1402] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:08.583129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7ccc2-6a63-4bbc-9f16-ee1ef5f791a4 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:08.584677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f0c7ccc2-6a63-4bbc-9f16-ee1ef5f791a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:08.736973 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:08.809488 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7ccc2-6a63-4bbc-9f16-ee1ef5f791a4 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:08.958989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11d000-ae80-47f0-b89e-a0bbf659e235 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:08.959708 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11d000-ae80-47f0-b89e-a0bbf659e235 service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:08.959987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11d000-ae80-47f0-b89e-a0bbf659e235 service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:08.960260 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11d000-ae80-47f0-b89e-a0bbf659e235 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:08.961996 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5a11d000-ae80-47f0-b89e-a0bbf659e235 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:09.073890 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:09.130712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11d000-ae80-47f0-b89e-a0bbf659e235 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:09.133727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11d000-ae80-47f0-b89e-a0bbf659e235 service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:09.170795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 707/1403] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:12.058217 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329789c3-53d4-4f64-99f8-cc979e96f392 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:12.058834 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329789c3-53d4-4f64-99f8-cc979e96f392 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:12.058834 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329789c3-53d4-4f64-99f8-cc979e96f392 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:12.060505 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 697/1404] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:12.066774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e99e2-0850-4fec-aab7-c4690ce996a6 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:12.067302 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e99e2-0850-4fec-aab7-c4690ce996a6 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:12.067414 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e99e2-0850-4fec-aab7-c4690ce996a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:12.141841 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-a39e99e2-0850-4fec-aab7-c4690ce996a6 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:12.143108 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-a39e99e2-0850-4fec-aab7-c4690ce996a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:12.284398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 708/1405] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:12.582530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329789c3-53d4-4f64-99f8-cc979e96f392 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:12.584162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-329789c3-53d4-4f64-99f8-cc979e96f392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:12.661743 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:12.797479 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329789c3-53d4-4f64-99f8-cc979e96f392 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:12.966286 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827446b-8cd6-4f5d-a87a-2f2a33c32d94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:12.966938 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827446b-8cd6-4f5d-a87a-2f2a33c32d94 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:12.967205 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827446b-8cd6-4f5d-a87a-2f2a33c32d94 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:12.967469 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827446b-8cd6-4f5d-a87a-2f2a33c32d94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:12.969751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1827446b-8cd6-4f5d-a87a-2f2a33c32d94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:13.001839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:13.135057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827446b-8cd6-4f5d-a87a-2f2a33c32d94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:13.138371 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827446b-8cd6-4f5d-a87a-2f2a33c32d94 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:13.180854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 698/1406] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:15.565249 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08f351-8e5f-48e9-9c9f-6855e3c638b1 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:15.565842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08f351-8e5f-48e9-9c9f-6855e3c638b1 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:15.565842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08f351-8e5f-48e9-9c9f-6855e3c638b1 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:15.567158 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 709/1407] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:15.573289 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32023c-dc51-4f21-a2f7-f68dceeeaaaa None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:15.574042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32023c-dc51-4f21-a2f7-f68dceeeaaaa None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:15.574042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32023c-dc51-4f21-a2f7-f68dceeeaaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:15.657821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-2a32023c-dc51-4f21-a2f7-f68dceeeaaaa None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:15.659296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2a32023c-dc51-4f21-a2f7-f68dceeeaaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:15.784465 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:15.788192 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 699/1408] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:37:16.098264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08f351-8e5f-48e9-9c9f-6855e3c638b1 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:16.099948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ef08f351-8e5f-48e9-9c9f-6855e3c638b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:16.155762 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:16.342537 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08f351-8e5f-48e9-9c9f-6855e3c638b1 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:16.484838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bcad9-95e9-4d88-b355-5a24e677ffa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:16.485485 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bcad9-95e9-4d88-b355-5a24e677ffa8 service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:16.485778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bcad9-95e9-4d88-b355-5a24e677ffa8 service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:16.486038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bcad9-95e9-4d88-b355-5a24e677ffa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:16.487719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-897bcad9-95e9-4d88-b355-5a24e677ffa8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:16.491583 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:16.604229 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897bcad9-95e9-4d88-b355-5a24e677ffa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:16.606841 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897bcad9-95e9-4d88-b355-5a24e677ffa8 service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:16.628906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 710/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:25.798323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:26.626754 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=43 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:28.912153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049713da-f710-4aaa-905c-733db9236145 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:28.912753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049713da-f710-4aaa-905c-733db9236145 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:28.912753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049713da-f710-4aaa-905c-733db9236145 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:28.914209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 700/1410] 10.222.0.57 () {54 vars in 918 bytes} [Mon Sep 26 05:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:28.919825 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c595d11-e862-40f9-94e6-0e1c5f20c30c None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:28.920321 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c595d11-e862-40f9-94e6-0e1c5f20c30c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:28.920403 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c595d11-e862-40f9-94e6-0e1c5f20c30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:29.006541 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-1c595d11-e862-40f9-94e6-0e1c5f20c30c None None] MFA Rules not processed for user `67fcd65eeb994b2381b83480e030f2ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:29.007898 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-1c595d11-e862-40f9-94e6-0e1c5f20c30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:29.131025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 711/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Sep 26 05:37:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:29.146360 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89550951-361e-408d-bb16-8ca6a98188f3 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:29.146670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89550951-361e-408d-bb16-8ca6a98188f3 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:29.146939 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89550951-361e-408d-bb16-8ca6a98188f3 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:29.148472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 701/1412] 10.222.0.57 () {54 vars in 918 bytes} [Mon Sep 26 05:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:29.153905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013d0c6-1a05-4882-bc18-46adde6846d0 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:29.154172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013d0c6-1a05-4882-bc18-46adde6846d0 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:29.154468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013d0c6-1a05-4882-bc18-46adde6846d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:29.257669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:29.260841 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-a013d0c6-1a05-4882-bc18-46adde6846d0 None None] MFA Rules not processed for user `92556048cbd04c409c3f4b6fd645627f`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:29.262680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-a013d0c6-1a05-4882-bc18-46adde6846d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:29.413259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 712/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Mon Sep 26 05:37:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:32.097453 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591263aa-76f4-43d6-9659-542bf85f6d35 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:32.098087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591263aa-76f4-43d6-9659-542bf85f6d35 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:32.098087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591263aa-76f4-43d6-9659-542bf85f6d35 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:32.099591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:37:32.106572 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b292b-b467-4d8b-ae8b-3cb359f5c4a5 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:32.108412 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b292b-b467-4d8b-ae8b-3cb359f5c4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:32.108412 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b292b-b467-4d8b-ae8b-3cb359f5c4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:32.201056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-593b292b-b467-4d8b-ae8b-3cb359f5c4a5 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:32.202355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-593b292b-b467-4d8b-ae8b-3cb359f5c4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:32.268903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:32.329757 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:32.620998 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591263aa-76f4-43d6-9659-542bf85f6d35 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:32.623581 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-591263aa-76f4-43d6-9659-542bf85f6d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:32.794369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591263aa-76f4-43d6-9659-542bf85f6d35 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:32.874279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:32.916274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc896ff-64d2-4bb6-95a4-0459361d74f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:32.916883 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc896ff-64d2-4bb6-95a4-0459361d74f9 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:32.917185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc896ff-64d2-4bb6-95a4-0459361d74f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:32.917459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc896ff-64d2-4bb6-95a4-0459361d74f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:32.919146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bbc896ff-64d2-4bb6-95a4-0459361d74f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:33.065320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc896ff-64d2-4bb6-95a4-0459361d74f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:33.066943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc896ff-64d2-4bb6-95a4-0459361d74f9 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:33.094293 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 703/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:34.272926 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605977c-4369-4bf4-815d-b91ffedceaaf None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:34.273529 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605977c-4369-4bf4-815d-b91ffedceaaf None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:34.273529 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605977c-4369-4bf4-815d-b91ffedceaaf None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:34.274864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:34.281719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea966e-48c2-4727-beae-4730e0492387 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:34.282244 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea966e-48c2-4727-beae-4730e0492387 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:34.282244 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea966e-48c2-4727-beae-4730e0492387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:34.362466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f5ea966e-48c2-4727-beae-4730e0492387 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:34.363699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f5ea966e-48c2-4727-beae-4730e0492387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:34.391009 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:34.472392 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:37:34.627287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2605977c-4369-4bf4-815d-b91ffedceaaf None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:34.628886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2605977c-4369-4bf4-815d-b91ffedceaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:34.807305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2605977c-4369-4bf4-815d-b91ffedceaaf None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:34.858230 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:34.948000 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478de2ef-f610-413e-9cda-28837a55045f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:34.948625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478de2ef-f610-413e-9cda-28837a55045f service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:34.948904 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478de2ef-f610-413e-9cda-28837a55045f service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:34.949210 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478de2ef-f610-413e-9cda-28837a55045f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:34.950893 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-478de2ef-f610-413e-9cda-28837a55045f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:35.062602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478de2ef-f610-413e-9cda-28837a55045f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:35.064220 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478de2ef-f610-413e-9cda-28837a55045f service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:35.084020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 715/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:36.241700 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aabbca-bbd8-4cc9-98e8-0ad55f3b3031 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:36.242236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aabbca-bbd8-4cc9-98e8-0ad55f3b3031 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:36.242236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aabbca-bbd8-4cc9-98e8-0ad55f3b3031 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:36.243653 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:36.250628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e8685-7836-419a-b13b-8c561d2bf34b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:36.251148 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e8685-7836-419a-b13b-8c561d2bf34b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:36.251148 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e8685-7836-419a-b13b-8c561d2bf34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:36.312524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:36.335287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-771e8685-7836-419a-b13b-8c561d2bf34b None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:36.336539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-771e8685-7836-419a-b13b-8c561d2bf34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:36.439267 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:36.588532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aabbca-bbd8-4cc9-98e8-0ad55f3b3031 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:36.590140 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-58aabbca-bbd8-4cc9-98e8-0ad55f3b3031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:36.746387 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aabbca-bbd8-4cc9-98e8-0ad55f3b3031 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:36.759591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:36.871762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bff8a1-5ea2-4d98-8db7-b742c362ec70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:36.872378 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bff8a1-5ea2-4d98-8db7-b742c362ec70 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:36.872655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bff8a1-5ea2-4d98-8db7-b742c362ec70 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:36.872928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bff8a1-5ea2-4d98-8db7-b742c362ec70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:36.874566 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d0bff8a1-5ea2-4d98-8db7-b742c362ec70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:36.971505 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bff8a1-5ea2-4d98-8db7-b742c362ec70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:36.973128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bff8a1-5ea2-4d98-8db7-b742c362ec70 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:36.992951 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 706/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:38.113421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296365f-5bfb-4a9c-a5f4-4f764108073b None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:38.114042 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296365f-5bfb-4a9c-a5f4-4f764108073b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:38.114042 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296365f-5bfb-4a9c-a5f4-4f764108073b None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:38.115422 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:38.123211 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faab877-9b53-4f6a-b2cc-310e9a429842 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:38.123729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faab877-9b53-4f6a-b2cc-310e9a429842 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:38.124908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faab877-9b53-4f6a-b2cc-310e9a429842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:38.158031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:38.220941 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-1faab877-9b53-4f6a-b2cc-310e9a429842 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:38.222196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1faab877-9b53-4f6a-b2cc-310e9a429842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:38.338758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:38.489339 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296365f-5bfb-4a9c-a5f4-4f764108073b None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:38.490846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5296365f-5bfb-4a9c-a5f4-4f764108073b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:38.622277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:38.665534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296365f-5bfb-4a9c-a5f4-4f764108073b None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:38.787378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d5df4-c314-4fa0-bf20-dc78fe935c48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:38.787896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d5df4-c314-4fa0-bf20-dc78fe935c48 service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:38.788170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d5df4-c314-4fa0-bf20-dc78fe935c48 service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:38.788464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d5df4-c314-4fa0-bf20-dc78fe935c48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:38.790285 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-081d5df4-c314-4fa0-bf20-dc78fe935c48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:38.877054 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:38.891773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d5df4-c314-4fa0-bf20-dc78fe935c48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:38.893442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d5df4-c314-4fa0-bf20-dc78fe935c48 service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:38.912772 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 718/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:40.067078 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627beb7-d86a-4a01-8e05-39a4fa12a224 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:40.067690 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627beb7-d86a-4a01-8e05-39a4fa12a224 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:40.067690 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627beb7-d86a-4a01-8e05-39a4fa12a224 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:40.069000 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:40.076251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c4c4c-ba3b-40ac-aeb4-2db46de2d880 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:40.076734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c4c4c-ba3b-40ac-aeb4-2db46de2d880 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:40.076858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c4c4c-ba3b-40ac-aeb4-2db46de2d880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:40.161379 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-1d9c4c4c-ba3b-40ac-aeb4-2db46de2d880 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:40.162649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-1d9c4c4c-ba3b-40ac-aeb4-2db46de2d880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:40.267043 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:40.416043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627beb7-d86a-4a01-8e05-39a4fa12a224 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:40.417666 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5627beb7-d86a-4a01-8e05-39a4fa12a224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:40.511184 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:40.587666 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627beb7-d86a-4a01-8e05-39a4fa12a224 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:40.702234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6910591b-40b7-4452-ad7d-5be1eb6a4afe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:40.702752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910591b-40b7-4452-ad7d-5be1eb6a4afe service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:40.703023 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910591b-40b7-4452-ad7d-5be1eb6a4afe service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:40.703289 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6910591b-40b7-4452-ad7d-5be1eb6a4afe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:40.704909 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6910591b-40b7-4452-ad7d-5be1eb6a4afe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:40.757521 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:40.801482 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6910591b-40b7-4452-ad7d-5be1eb6a4afe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:40.803096 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6910591b-40b7-4452-ad7d-5be1eb6a4afe service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:40.822299 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 709/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:41.893057 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c29d45-ee21-4dc7-8a90-cbb6dfda7517 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:41.893722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c29d45-ee21-4dc7-8a90-cbb6dfda7517 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:41.893722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c29d45-ee21-4dc7-8a90-cbb6dfda7517 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:41.895001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:41.902023 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d93d8-46a9-4fce-a86e-4d5b3949f8f5 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:41.902542 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d93d8-46a9-4fce-a86e-4d5b3949f8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:41.902542 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d93d8-46a9-4fce-a86e-4d5b3949f8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:41.988129 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-628d93d8-46a9-4fce-a86e-4d5b3949f8f5 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:41.989419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-628d93d8-46a9-4fce-a86e-4d5b3949f8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:42.093987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:37:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:42.245230 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c29d45-ee21-4dc7-8a90-cbb6dfda7517 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:42.246918 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-61c29d45-ee21-4dc7-8a90-cbb6dfda7517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:42.299117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:42.403459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c29d45-ee21-4dc7-8a90-cbb6dfda7517 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:42.525553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e319b3-ec79-4a89-934b-7d62470d7325 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:42.526069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e319b3-ec79-4a89-934b-7d62470d7325 service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:42.526375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e319b3-ec79-4a89-934b-7d62470d7325 service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:42.526649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e319b3-ec79-4a89-934b-7d62470d7325 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:42.528287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-73e319b3-ec79-4a89-934b-7d62470d7325 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:42.535794 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:42.752858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e319b3-ec79-4a89-934b-7d62470d7325 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:42.754521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e319b3-ec79-4a89-934b-7d62470d7325 service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:42.773707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 721/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:43.844370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c989f25-686e-47f8-982c-85f946466bd2 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:43.844971 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c989f25-686e-47f8-982c-85f946466bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:43.844971 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c989f25-686e-47f8-982c-85f946466bd2 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:43.846326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:43.853580 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5939d3-231a-4273-bbee-c3792abab4e0 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:43.854187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5939d3-231a-4273-bbee-c3792abab4e0 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:43.854187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5939d3-231a-4273-bbee-c3792abab4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:43.949672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-4b5939d3-231a-4273-bbee-c3792abab4e0 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:43.950886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4b5939d3-231a-4273-bbee-c3792abab4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:44.021638 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:44.050373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:44.198534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c989f25-686e-47f8-982c-85f946466bd2 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:44.199904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6c989f25-686e-47f8-982c-85f946466bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:44.220277 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:44.354071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c989f25-686e-47f8-982c-85f946466bd2 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:44.450922 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:44.480492 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb968f-51c0-4fda-bf94-1ca54a12fee8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:44.481104 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb968f-51c0-4fda-bf94-1ca54a12fee8 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:44.481416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb968f-51c0-4fda-bf94-1ca54a12fee8 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:44.481691 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb968f-51c0-4fda-bf94-1ca54a12fee8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:44.483340 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-44fb968f-51c0-4fda-bf94-1ca54a12fee8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:44.581570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb968f-51c0-4fda-bf94-1ca54a12fee8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:44.583162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb968f-51c0-4fda-bf94-1ca54a12fee8 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:44.601814 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 712/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:45.658273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0f9a6-675d-4a44-9dfb-cc732a7b2cae None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:45.658922 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0f9a6-675d-4a44-9dfb-cc732a7b2cae None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:45.658922 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0f9a6-675d-4a44-9dfb-cc732a7b2cae None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:45.660225 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:45.667029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ddc5d-68be-4fb5-a97c-8562d50f5319 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:45.667560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ddc5d-68be-4fb5-a97c-8562d50f5319 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:45.667560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ddc5d-68be-4fb5-a97c-8562d50f5319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:45.751906 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ca1ddc5d-68be-4fb5-a97c-8562d50f5319 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:45.753180 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ca1ddc5d-68be-4fb5-a97c-8562d50f5319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:45.802393 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:45.861567 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:46.014519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0f9a6-675d-4a44-9dfb-cc732a7b2cae None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:46.016011 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f9d0f9a6-675d-4a44-9dfb-cc732a7b2cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:46.174039 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0f9a6-675d-4a44-9dfb-cc732a7b2cae None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:46.231162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:46.291637 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991458c3-3a23-4912-935b-f8ff4f90d813 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:46.292152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991458c3-3a23-4912-935b-f8ff4f90d813 service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:46.292421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991458c3-3a23-4912-935b-f8ff4f90d813 service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:46.292671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991458c3-3a23-4912-935b-f8ff4f90d813 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:46.294299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-991458c3-3a23-4912-935b-f8ff4f90d813 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:46.388598 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991458c3-3a23-4912-935b-f8ff4f90d813 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:46.391161 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991458c3-3a23-4912-935b-f8ff4f90d813 service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:46.413151 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 724/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:47.484999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ddd3c-8572-4699-b8c3-4d99aa7634b0 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:47.485616 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ddd3c-8572-4699-b8c3-4d99aa7634b0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:47.485616 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ddd3c-8572-4699-b8c3-4d99aa7634b0 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:47.486822 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:47.493839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739bb86-804a-4fa1-bc03-8107e3a555da None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:47.494375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739bb86-804a-4fa1-bc03-8107e3a555da None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:47.494375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739bb86-804a-4fa1-bc03-8107e3a555da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:47.573823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-e739bb86-804a-4fa1-bc03-8107e3a555da None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:47.575056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-e739bb86-804a-4fa1-bc03-8107e3a555da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:47.577156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:47.679343 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:47.828219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ddd3c-8572-4699-b8c3-4d99aa7634b0 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:47.829818 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5d8ddd3c-8572-4699-b8c3-4d99aa7634b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:47.996069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ddd3c-8572-4699-b8c3-4d99aa7634b0 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:48.026176 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:48.123427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e745df-4bcc-48c1-9f85-1c742d547531 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:48.123951 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e745df-4bcc-48c1-9f85-1c742d547531 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:48.124233 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e745df-4bcc-48c1-9f85-1c742d547531 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:48.124503 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e745df-4bcc-48c1-9f85-1c742d547531 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:48.126346 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e8e745df-4bcc-48c1-9f85-1c742d547531 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:48.225385 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e745df-4bcc-48c1-9f85-1c742d547531 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:48.227131 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e745df-4bcc-48c1-9f85-1c742d547531 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:48.245566 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 715/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:50.382251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d182c-27cf-4d01-a0ff-4cd05ede33a1 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:50.382865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d182c-27cf-4d01-a0ff-4cd05ede33a1 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:50.382865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d182c-27cf-4d01-a0ff-4cd05ede33a1 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:50.384138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:37:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:50.390329 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b8971-2bae-4e7e-84c0-daf7b5625265 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:50.390803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b8971-2bae-4e7e-84c0-daf7b5625265 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:50.390920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b8971-2bae-4e7e-84c0-daf7b5625265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:50.444505 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:50.474676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ba9b8971-2bae-4e7e-84c0-daf7b5625265 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:50.476279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ba9b8971-2bae-4e7e-84c0-daf7b5625265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:50.582214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:50.868429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d182c-27cf-4d01-a0ff-4cd05ede33a1 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:50.870008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-261d182c-27cf-4d01-a0ff-4cd05ede33a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:50.976977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d182c-27cf-4d01-a0ff-4cd05ede33a1 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:51.010112 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:51.054221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d9a9d-e576-4d8d-b10d-39b59a51c158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:51.054830 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d9a9d-e576-4d8d-b10d-39b59a51c158 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:51.055122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d9a9d-e576-4d8d-b10d-39b59a51c158 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:51.055353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d9a9d-e576-4d8d-b10d-39b59a51c158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:51.056957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-bb9d9a9d-e576-4d8d-b10d-39b59a51c158 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:51.172001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d9a9d-e576-4d8d-b10d-39b59a51c158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:51.173357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d9a9d-e576-4d8d-b10d-39b59a51c158 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:51.192183 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 727/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:53.572501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe12d3-a28a-4012-851e-4060bb13036a None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:53.573109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe12d3-a28a-4012-851e-4060bb13036a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:53.573109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe12d3-a28a-4012-851e-4060bb13036a None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:53.574401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:53.580080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c8786-5173-4ad6-9598-fc3d3abf071b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:53.580521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c8786-5173-4ad6-9598-fc3d3abf071b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:53.580632 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c8786-5173-4ad6-9598-fc3d3abf071b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:53.650759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-190c8786-5173-4ad6-9598-fc3d3abf071b None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:53.651987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-190c8786-5173-4ad6-9598-fc3d3abf071b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:53.666303 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:53.760557 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:54.048510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe12d3-a28a-4012-851e-4060bb13036a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:54.050104 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-abfe12d3-a28a-4012-851e-4060bb13036a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:54.165417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe12d3-a28a-4012-851e-4060bb13036a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:54.183783 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:54.266698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609dbd3-4da8-4ede-870a-6dd22a097504 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:54.267319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609dbd3-4da8-4ede-870a-6dd22a097504 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:54.267598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609dbd3-4da8-4ede-870a-6dd22a097504 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:54.267868 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609dbd3-4da8-4ede-870a-6dd22a097504 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:54.269561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9609dbd3-4da8-4ede-870a-6dd22a097504 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:54.391076 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609dbd3-4da8-4ede-870a-6dd22a097504 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:54.392524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609dbd3-4da8-4ede-870a-6dd22a097504 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:54.411626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 718/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:37:56.921087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f102e1a-76f7-4e51-9905-288405d88240 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:56.921724 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f102e1a-76f7-4e51-9905-288405d88240 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:56.921724 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f102e1a-76f7-4e51-9905-288405d88240 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:56.923013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:37:56.928678 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2108f-30e9-43a3-95aa-90ead1ce9ee6 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:56.929166 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2108f-30e9-43a3-95aa-90ead1ce9ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:56.929282 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2108f-30e9-43a3-95aa-90ead1ce9ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:56.979858 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:57.000961 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a7a2108f-30e9-43a3-95aa-90ead1ce9ee6 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:37:57.002690 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a7a2108f-30e9-43a3-95aa-90ead1ce9ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:57.111241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:37:57.390973 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f102e1a-76f7-4e51-9905-288405d88240 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:37:57.392538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6f102e1a-76f7-4e51-9905-288405d88240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:57.511777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f102e1a-76f7-4e51-9905-288405d88240 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:37:57.553864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:37:57.587689 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d61472-0726-421e-b2c7-b191c02d2d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:37:57.588291 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d61472-0726-421e-b2c7-b191c02d2d9e service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:37:57.588563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d61472-0726-421e-b2c7-b191c02d2d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:37:57.588817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d61472-0726-421e-b2c7-b191c02d2d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:37:57.590442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-96d61472-0726-421e-b2c7-b191c02d2d9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:37:57.707094 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d61472-0726-421e-b2c7-b191c02d2d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:37:57.708441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d61472-0726-421e-b2c7-b191c02d2d9e service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:37:57.727734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 730/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:37:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:00.037985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f602202-12ab-48c2-b0e1-df85c1829c7e None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:00.038550 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f602202-12ab-48c2-b0e1-df85c1829c7e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:00.038550 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f602202-12ab-48c2-b0e1-df85c1829c7e None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:00.039777 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:38:00.045505 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9d6aa-986a-4c5d-a625-12dd9989b86d None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:00.045944 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9d6aa-986a-4c5d-a625-12dd9989b86d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:00.046038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9d6aa-986a-4c5d-a625-12dd9989b86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:00.123012 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-92f9d6aa-986a-4c5d-a625-12dd9989b86d None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:00.124512 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-92f9d6aa-986a-4c5d-a625-12dd9989b86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:00.147861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:00.226296 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:00.509012 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f602202-12ab-48c2-b0e1-df85c1829c7e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:00.510638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0f602202-12ab-48c2-b0e1-df85c1829c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:00.622169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f602202-12ab-48c2-b0e1-df85c1829c7e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:00.644796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:00.702529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb4a4b-5df2-47d0-8f61-5edb073a9a33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:00.703149 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb4a4b-5df2-47d0-8f61-5edb073a9a33 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:00.703425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb4a4b-5df2-47d0-8f61-5edb073a9a33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:00.703694 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb4a4b-5df2-47d0-8f61-5edb073a9a33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:00.705343 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7aeb4a4b-5df2-47d0-8f61-5edb073a9a33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:00.822961 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb4a4b-5df2-47d0-8f61-5edb073a9a33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:00.824426 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb4a4b-5df2-47d0-8f61-5edb073a9a33 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:00.846118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 721/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:03.071743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37809e1-76bd-4bdd-bd24-e3343e270525 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:03.072336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37809e1-76bd-4bdd-bd24-e3343e270525 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:03.072336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37809e1-76bd-4bdd-bd24-e3343e270525 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:03.073680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:38:03.079523 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63051cc9-cb7a-489e-996e-1f5f0a68295c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:03.079987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63051cc9-cb7a-489e-996e-1f5f0a68295c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:03.080129 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63051cc9-cb7a-489e-996e-1f5f0a68295c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:03.147586 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:03.150443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-63051cc9-cb7a-489e-996e-1f5f0a68295c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:03.152342 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-63051cc9-cb7a-489e-996e-1f5f0a68295c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:03.254770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:38:03.531040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37809e1-76bd-4bdd-bd24-e3343e270525 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:03.532642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f37809e1-76bd-4bdd-bd24-e3343e270525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:03.643369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37809e1-76bd-4bdd-bd24-e3343e270525 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:03.700833 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:03.727694 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86806d-8783-4ad5-bf2e-c30cb957c740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:03.728318 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86806d-8783-4ad5-bf2e-c30cb957c740 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:03.728592 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86806d-8783-4ad5-bf2e-c30cb957c740 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:03.728843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86806d-8783-4ad5-bf2e-c30cb957c740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:03.730593 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2b86806d-8783-4ad5-bf2e-c30cb957c740 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:03.852263 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86806d-8783-4ad5-bf2e-c30cb957c740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:03.853752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86806d-8783-4ad5-bf2e-c30cb957c740 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:03.874503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 733/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:06.260277 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d75bab-ff8a-435a-95e1-1765776b62eb None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:06.260871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d75bab-ff8a-435a-95e1-1765776b62eb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:06.260871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d75bab-ff8a-435a-95e1-1765776b62eb None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:06.262247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:38:06.268554 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0be84b-ac37-4267-a50d-081cd31007a4 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:06.269029 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0be84b-ac37-4267-a50d-081cd31007a4 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:06.269168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0be84b-ac37-4267-a50d-081cd31007a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:06.341021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-9e0be84b-ac37-4267-a50d-081cd31007a4 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:06.342272 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9e0be84b-ac37-4267-a50d-081cd31007a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:06.374226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:06.446685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:06.735991 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d75bab-ff8a-435a-95e1-1765776b62eb None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:06.737772 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-67d75bab-ff8a-435a-95e1-1765776b62eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:06.852055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d75bab-ff8a-435a-95e1-1765776b62eb None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:06.885996 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:06.938965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6726a9-e231-45fb-b479-a97d305f7e59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:06.939573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6726a9-e231-45fb-b479-a97d305f7e59 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:06.939851 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6726a9-e231-45fb-b479-a97d305f7e59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:06.940189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6726a9-e231-45fb-b479-a97d305f7e59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:06.942165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ea6726a9-e231-45fb-b479-a97d305f7e59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:07.063344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6726a9-e231-45fb-b479-a97d305f7e59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:07.064783 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6726a9-e231-45fb-b479-a97d305f7e59 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:07.085963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 724/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:09.315617 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6d1ae-d367-4091-bca7-1cffb01f3c0a None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:09.316252 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6d1ae-d367-4091-bca7-1cffb01f3c0a None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:09.316252 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6d1ae-d367-4091-bca7-1cffb01f3c0a None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:09.317573 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:38:09.323107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841238ad-edcc-4a9d-b0ec-b8326ec000d2 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:09.323668 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841238ad-edcc-4a9d-b0ec-b8326ec000d2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:09.323668 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841238ad-edcc-4a9d-b0ec-b8326ec000d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:09.391318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-841238ad-edcc-4a9d-b0ec-b8326ec000d2 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:09.392648 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-841238ad-edcc-4a9d-b0ec-b8326ec000d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:09.401431 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:09.493998 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:09.775281 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6d1ae-d367-4091-bca7-1cffb01f3c0a None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:09.776867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-72d6d1ae-d367-4091-bca7-1cffb01f3c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:09.885299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6d1ae-d367-4091-bca7-1cffb01f3c0a None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:09.943478 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:09.962892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d20eb-f0b9-4d04-a5c8-94f93323bae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:09.963507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d20eb-f0b9-4d04-a5c8-94f93323bae5 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:09.963786 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d20eb-f0b9-4d04-a5c8-94f93323bae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:09.964056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d20eb-f0b9-4d04-a5c8-94f93323bae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:09.965854 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c90d20eb-f0b9-4d04-a5c8-94f93323bae5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:10.087004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d20eb-f0b9-4d04-a5c8-94f93323bae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:10.088459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d20eb-f0b9-4d04-a5c8-94f93323bae5 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:10.108526 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 736/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:12.352333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79be05-5a82-4c2a-936e-41e5361c1444 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:12.352907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79be05-5a82-4c2a-936e-41e5361c1444 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:12.353036 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79be05-5a82-4c2a-936e-41e5361c1444 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:12.354371 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:38:12.359940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951d19b-1619-4ff9-91bc-296f205c6b47 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:12.360355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951d19b-1619-4ff9-91bc-296f205c6b47 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:12.360495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951d19b-1619-4ff9-91bc-296f205c6b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:12.428418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-9951d19b-1619-4ff9-91bc-296f205c6b47 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:12.429723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9951d19b-1619-4ff9-91bc-296f205c6b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:12.469467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:12.529734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:12.812382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79be05-5a82-4c2a-936e-41e5361c1444 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:12.814006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ae79be05-5a82-4c2a-936e-41e5361c1444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:12.923224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79be05-5a82-4c2a-936e-41e5361c1444 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:12.964095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:13.002237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a1f13-02a6-4be3-9e98-e16768748e1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:13.002842 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a1f13-02a6-4be3-9e98-e16768748e1a service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:13.003118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a1f13-02a6-4be3-9e98-e16768748e1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:13.003399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a1f13-02a6-4be3-9e98-e16768748e1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:13.004988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ff1a1f13-02a6-4be3-9e98-e16768748e1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:13.119063 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a1f13-02a6-4be3-9e98-e16768748e1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:13.120495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a1f13-02a6-4be3-9e98-e16768748e1a service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:13.139088 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 727/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:15.352205 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c6c64-80e4-4500-acd5-01d295a615b9 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:15.352809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c6c64-80e4-4500-acd5-01d295a615b9 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:15.352809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c6c64-80e4-4500-acd5-01d295a615b9 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:15.354111 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:38:15.359912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e6a4b-6e1b-4dbb-b028-4f8308d4aae8 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:15.360344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e6a4b-6e1b-4dbb-b028-4f8308d4aae8 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:15.360491 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e6a4b-6e1b-4dbb-b028-4f8308d4aae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:15.432380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8a0e6a4b-6e1b-4dbb-b028-4f8308d4aae8 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:15.433710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8a0e6a4b-6e1b-4dbb-b028-4f8308d4aae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:15.453110 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:15.536147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:15.817418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c6c64-80e4-4500-acd5-01d295a615b9 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:15.819076 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-395c6c64-80e4-4500-acd5-01d295a615b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:15.930031 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c6c64-80e4-4500-acd5-01d295a615b9 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:16.002140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:16.009392 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be91564-0ace-4e05-8d8e-f9970f5ba803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:16.010008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be91564-0ace-4e05-8d8e-f9970f5ba803 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:16.010299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be91564-0ace-4e05-8d8e-f9970f5ba803 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:16.010568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be91564-0ace-4e05-8d8e-f9970f5ba803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:16.012462 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-1be91564-0ace-4e05-8d8e-f9970f5ba803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:16.127523 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be91564-0ace-4e05-8d8e-f9970f5ba803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:16.130225 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be91564-0ace-4e05-8d8e-f9970f5ba803 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:16.149460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 739/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:18.341898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19d39b-13cf-4cc8-9b93-9892d1078352 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:18.342854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19d39b-13cf-4cc8-9b93-9892d1078352 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:18.342854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19d39b-13cf-4cc8-9b93-9892d1078352 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:18.344263 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:38:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:38:18.350893 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1faad-2d0f-44f6-bfb7-2e82d53a1aeb None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:18.351567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1faad-2d0f-44f6-bfb7-2e82d53a1aeb None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:18.351567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1faad-2d0f-44f6-bfb7-2e82d53a1aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:18.420859 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d9f1faad-2d0f-44f6-bfb7-2e82d53a1aeb None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:18.422102 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d9f1faad-2d0f-44f6-bfb7-2e82d53a1aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:18.475666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:18.532311 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:18.818163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19d39b-13cf-4cc8-9b93-9892d1078352 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:18.820009 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-df19d39b-13cf-4cc8-9b93-9892d1078352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:18.925976 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19d39b-13cf-4cc8-9b93-9892d1078352 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:18.976959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:19.003598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97b021-e679-41b9-9d42-7b188872398f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:19.004205 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97b021-e679-41b9-9d42-7b188872398f service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:19.004484 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97b021-e679-41b9-9d42-7b188872398f service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:19.004740 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97b021-e679-41b9-9d42-7b188872398f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:19.006490 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9c97b021-e679-41b9-9d42-7b188872398f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:19.122513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97b021-e679-41b9-9d42-7b188872398f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:19.123869 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97b021-e679-41b9-9d42-7b188872398f service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:19.145486 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 730/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:28.530675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=22 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:29.142875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=66 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:39.142227 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f37a4a-6172-455e-be67-8c2d2caf6d2b None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:39.142802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f37a4a-6172-455e-be67-8c2d2caf6d2b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:39.142802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f37a4a-6172-455e-be67-8c2d2caf6d2b None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:39.144039 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 741/1471] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:38:39.149785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e7d6e-5fbd-4a74-b0dc-77685ee432f5 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:39.150290 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e7d6e-5fbd-4a74-b0dc-77685ee432f5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:39.150414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e7d6e-5fbd-4a74-b0dc-77685ee432f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:39.223568 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a98e7d6e-5fbd-4a74-b0dc-77685ee432f5 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:39.224970 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a98e7d6e-5fbd-4a74-b0dc-77685ee432f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:39.334901 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 731/1472] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:39.632012 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b642d2e-bb9d-45af-ba31-5e7272bee2e7 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:39.632300 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b642d2e-bb9d-45af-ba31-5e7272bee2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:39.632576 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b642d2e-bb9d-45af-ba31-5e7272bee2e7 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:39.633937 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 742/1473] 10.222.0.57 () {54 vars in 962 bytes} [Mon Sep 26 05:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:38:39.639181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62221a4-56ec-4991-8c1e-4f70f8703dd9 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:39.639470 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62221a4-56ec-4991-8c1e-4f70f8703dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:39.639736 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62221a4-56ec-4991-8c1e-4f70f8703dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:39.720165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-e62221a4-56ec-4991-8c1e-4f70f8703dd9 None None] MFA Rules not processed for user `90c0fabecff2457e931c8c1c36681197`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:39.721413 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e62221a4-56ec-4991-8c1e-4f70f8703dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:39.744151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:39.805637 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 732/1474] 10.222.0.57 () {58 vars in 1059 bytes} [Mon Sep 26 05:38:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:39.813731 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b642d2e-bb9d-45af-ba31-5e7272bee2e7 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:39.815268 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-1b642d2e-bb9d-45af-ba31-5e7272bee2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:39.926334 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b642d2e-bb9d-45af-ba31-5e7272bee2e7 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:40.002912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9d07c-318e-4aec-9d08-d509e527f36c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c0fabecff2457e931c8c1c36681197', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:40.003520 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9d07c-318e-4aec-9d08-d509e527f36c service glance] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:40.003797 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9d07c-318e-4aec-9d08-d509e527f36c service glance] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:40.004063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9d07c-318e-4aec-9d08-d509e527f36c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:40.005784 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-35a9d07c-318e-4aec-9d08-d509e527f36c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:40.064707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:40.123286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9d07c-318e-4aec-9d08-d509e527f36c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:40.124983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9d07c-318e-4aec-9d08-d509e527f36c service glance] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:40.144406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 743/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:42.381577 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de41535-731f-4a58-a110-6898327299ca None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:42.382177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de41535-731f-4a58-a110-6898327299ca None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:42.382177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de41535-731f-4a58-a110-6898327299ca None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:42.383483 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 733/1476] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:38:42.389283 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f77bb2-bff6-41d7-8a78-66a4838c2779 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:42.389797 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f77bb2-bff6-41d7-8a78-66a4838c2779 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:42.389797 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f77bb2-bff6-41d7-8a78-66a4838c2779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:42.461651 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-83f77bb2-bff6-41d7-8a78-66a4838c2779 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:42.462888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-83f77bb2-bff6-41d7-8a78-66a4838c2779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:42.565503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 744/1477] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:42.836180 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de41535-731f-4a58-a110-6898327299ca None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:42.837693 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2de41535-731f-4a58-a110-6898327299ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:42.970922 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de41535-731f-4a58-a110-6898327299ca None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:43.023406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:43.071724 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc373e-4a34-4cb3-8813-b5bedf98c466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:43.072340 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc373e-4a34-4cb3-8813-b5bedf98c466 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:43.072604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc373e-4a34-4cb3-8813-b5bedf98c466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:43.072871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc373e-4a34-4cb3-8813-b5bedf98c466 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:43.073571 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc373e-4a34-4cb3-8813-b5bedf98c466 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:43.075381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: WARNING py.warnings [None req-06dc373e-4a34-4cb3-8813-b5bedf98c466 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:38:43.075381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: warnings.warn(msg) Sep 26 05:38:43.075381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: Sep 26 05:38:43.076552 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc373e-4a34-4cb3-8813-b5bedf98c466 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:43.098247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 734/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Mon Sep 26 05:38:42 2022] GET /identity/v3/projects => generated 2553 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:38:45.306395 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5138a3e-75ee-47c5-9898-91c86ad687e1 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:45.307010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5138a3e-75ee-47c5-9898-91c86ad687e1 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:45.307010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5138a3e-75ee-47c5-9898-91c86ad687e1 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:45.308293 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 745/1479] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:38:45.314433 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a645a38-0feb-4a70-955e-5b18dc5c5110 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:45.314924 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a645a38-0feb-4a70-955e-5b18dc5c5110 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:45.315040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a645a38-0feb-4a70-955e-5b18dc5c5110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:45.384315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-5a645a38-0feb-4a70-955e-5b18dc5c5110 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:45.385599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5a645a38-0feb-4a70-955e-5b18dc5c5110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:45.491899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 735/1480] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:38:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:45.780968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5138a3e-75ee-47c5-9898-91c86ad687e1 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:45.782793 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c5138a3e-75ee-47c5-9898-91c86ad687e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:45.793544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:45.935580 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5138a3e-75ee-47c5-9898-91c86ad687e1 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:46.020706 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:46.053558 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bb4c1-ac79-4657-a6bc-28c597feb9d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:46.054182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bb4c1-ac79-4657-a6bc-28c597feb9d6 service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:46.054469 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bb4c1-ac79-4657-a6bc-28c597feb9d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:46.054770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bb4c1-ac79-4657-a6bc-28c597feb9d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:46.056420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-8c7bb4c1-ac79-4657-a6bc-28c597feb9d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:46.156936 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bb4c1-ac79-4657-a6bc-28c597feb9d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:46.158574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bb4c1-ac79-4657-a6bc-28c597feb9d6 service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:46.176515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 746/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:48.512877 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e30a9-4e6c-489f-8d4d-e0f8b7e51f91 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:48.513501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e30a9-4e6c-489f-8d4d-e0f8b7e51f91 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:48.513501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e30a9-4e6c-489f-8d4d-e0f8b7e51f91 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:48.514785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 736/1482] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:38:48.520871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d38202-816a-4f88-90e4-33a74f952024 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:48.521595 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d38202-816a-4f88-90e4-33a74f952024 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:48.521595 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d38202-816a-4f88-90e4-33a74f952024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:48.591924 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-44d38202-816a-4f88-90e4-33a74f952024 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:48.593172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-44d38202-816a-4f88-90e4-33a74f952024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:48.640250 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:48.695439 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 747/1483] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:48.981050 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e30a9-4e6c-489f-8d4d-e0f8b7e51f91 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:48.982732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-fa7e30a9-4e6c-489f-8d4d-e0f8b7e51f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:49.006946 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:49.089128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e30a9-4e6c-489f-8d4d-e0f8b7e51f91 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:49.168059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a60ba-c6a6-4d87-95ee-5862deeef16c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:49.168572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a60ba-c6a6-4d87-95ee-5862deeef16c service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:49.168847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a60ba-c6a6-4d87-95ee-5862deeef16c service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:49.169113 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a60ba-c6a6-4d87-95ee-5862deeef16c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:49.170754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f14a60ba-c6a6-4d87-95ee-5862deeef16c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:49.250955 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:49.287435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a60ba-c6a6-4d87-95ee-5862deeef16c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:49.288895 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a60ba-c6a6-4d87-95ee-5862deeef16c service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:49.308991 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 737/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:51.513588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeec4b6-d00a-402f-bab8-f384bb528881 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:51.514157 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeec4b6-d00a-402f-bab8-f384bb528881 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:51.514157 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeec4b6-d00a-402f-bab8-f384bb528881 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:51.515387 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 748/1485] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:38:51.521030 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438cc2a-4e9c-46f7-a18e-fbaef237edad None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:51.521559 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438cc2a-4e9c-46f7-a18e-fbaef237edad None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:51.521715 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438cc2a-4e9c-46f7-a18e-fbaef237edad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:51.592711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-6438cc2a-4e9c-46f7-a18e-fbaef237edad None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:51.593997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6438cc2a-4e9c-46f7-a18e-fbaef237edad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:51.697286 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 738/1486] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:51.983599 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeec4b6-d00a-402f-bab8-f384bb528881 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:51.985282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-baeec4b6-d00a-402f-bab8-f384bb528881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:52.096116 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeec4b6-d00a-402f-bab8-f384bb528881 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:52.172553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:52.174937 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0404a-4948-486a-be78-7f915098a872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:52.175658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0404a-4948-486a-be78-7f915098a872 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:52.176164 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0404a-4948-486a-be78-7f915098a872 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:52.176449 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0404a-4948-486a-be78-7f915098a872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:52.178679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-bbf0404a-4948-486a-be78-7f915098a872 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:52.297498 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0404a-4948-486a-be78-7f915098a872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:52.298945 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0404a-4948-486a-be78-7f915098a872 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:52.318328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 749/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:54.642813 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c50ac-78c0-46ed-a892-cc21bf313661 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:54.643425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c50ac-78c0-46ed-a892-cc21bf313661 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:54.643425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c50ac-78c0-46ed-a892-cc21bf313661 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:54.644694 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 739/1488] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:38:54.650474 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdedb9f-66ca-4552-b69b-0e329eb7a337 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:54.650935 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdedb9f-66ca-4552-b69b-0e329eb7a337 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:54.651077 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdedb9f-66ca-4552-b69b-0e329eb7a337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:54.719461 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-7cdedb9f-66ca-4552-b69b-0e329eb7a337 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:54.720667 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7cdedb9f-66ca-4552-b69b-0e329eb7a337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:54.783140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:54.828669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 750/1489] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:55.120391 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c50ac-78c0-46ed-a892-cc21bf313661 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:55.121982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a23c50ac-78c0-46ed-a892-cc21bf313661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:55.159794 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:55.230833 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c50ac-78c0-46ed-a892-cc21bf313661 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:55.311187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd147348-70d3-4056-b059-10b1d0a2a92f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:55.311799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd147348-70d3-4056-b059-10b1d0a2a92f service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:55.312080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd147348-70d3-4056-b059-10b1d0a2a92f service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:55.312351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd147348-70d3-4056-b059-10b1d0a2a92f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:55.314001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cd147348-70d3-4056-b059-10b1d0a2a92f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:55.398918 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:55.430503 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd147348-70d3-4056-b059-10b1d0a2a92f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:55.431867 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd147348-70d3-4056-b059-10b1d0a2a92f service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:55.450813 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 740/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:38:57.863713 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bbbf4-a36e-4872-a840-e9da194587f7 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:57.864335 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bbbf4-a36e-4872-a840-e9da194587f7 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:57.864335 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bbbf4-a36e-4872-a840-e9da194587f7 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:57.866248 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 751/1491] 10.222.0.57 () {54 vars in 927 bytes} [Mon Sep 26 05:38:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:38:57.872136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabec0e7-e3df-40c7-b7e4-188ae365ffd4 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:57.872596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabec0e7-e3df-40c7-b7e4-188ae365ffd4 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:57.872720 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabec0e7-e3df-40c7-b7e4-188ae365ffd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:57.944360 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-fabec0e7-e3df-40c7-b7e4-188ae365ffd4 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:38:57.945728 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-fabec0e7-e3df-40c7-b7e4-188ae365ffd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:58.045099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 741/1492] 10.222.0.57 () {58 vars in 1024 bytes} [Mon Sep 26 05:38:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:38:58.330563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356bbbf4-a36e-4872-a840-e9da194587f7 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:38:58.332138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-356bbbf4-a36e-4872-a840-e9da194587f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:58.444218 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356bbbf4-a36e-4872-a840-e9da194587f7 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:38:58.520913 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c72b0-fa4a-4fa5-8888-28624ba81fbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:38:58.521508 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c72b0-fa4a-4fa5-8888-28624ba81fbe service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:38:58.521786 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c72b0-fa4a-4fa5-8888-28624ba81fbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:38:58.522030 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c72b0-fa4a-4fa5-8888-28624ba81fbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:38:58.523653 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-1d5c72b0-fa4a-4fa5-8888-28624ba81fbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:38:58.533094 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:38:58.639014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c72b0-fa4a-4fa5-8888-28624ba81fbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:38:58.640440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c72b0-fa4a-4fa5-8888-28624ba81fbe service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:38:58.659372 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 752/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:38:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:39:08.043634 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=82 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:39:08.656897 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=50 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:40:33.726046 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d12c4-ffec-471c-8ae8-63c0e2c91abc None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:33.726681 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d12c4-ffec-471c-8ae8-63c0e2c91abc None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:33.726681 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d12c4-ffec-471c-8ae8-63c0e2c91abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:33.812880 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f11d12c4-ffec-471c-8ae8-63c0e2c91abc None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:40:33.814153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f11d12c4-ffec-471c-8ae8-63c0e2c91abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:33.923140 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 742/1494] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:40:33.932424 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3667805-8f51-4d98-96e1-e76157568951 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:33.932984 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3667805-8f51-4d98-96e1-e76157568951 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:33.932984 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3667805-8f51-4d98-96e1-e76157568951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:34.018590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-f3667805-8f51-4d98-96e1-e76157568951 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:40:34.019705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f3667805-8f51-4d98-96e1-e76157568951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:34.120727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 753/1495] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:40:34.125759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d12c4-ffec-471c-8ae8-63c0e2c91abc None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:34.135919 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d12c4-ffec-471c-8ae8-63c0e2c91abc None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:34.140756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d6881-2924-46be-a4f9-49fce8830650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:34.141391 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d6881-2924-46be-a4f9-49fce8830650 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:34.141661 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d6881-2924-46be-a4f9-49fce8830650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:34.141929 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d6881-2924-46be-a4f9-49fce8830650 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:34.142569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d6881-2924-46be-a4f9-49fce8830650 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:40:34.142866 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d6881-2924-46be-a4f9-49fce8830650 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:34.143880 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: WARNING py.warnings [None req-b99d6881-2924-46be-a4f9-49fce8830650 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:40:34.143880 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: warnings.warn(msg) Sep 26 05:40:34.143880 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: Sep 26 05:40:34.144758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d6881-2924-46be-a4f9-49fce8830650 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:34.157084 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 743/1496] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:40:34.165904 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3667805-8f51-4d98-96e1-e76157568951 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:34.176003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3667805-8f51-4d98-96e1-e76157568951 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:34.182128 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e646932-96a7-4023-8a5e-407fbb8487c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:34.182810 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e646932-96a7-4023-8a5e-407fbb8487c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:34.183089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e646932-96a7-4023-8a5e-407fbb8487c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:34.183366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e646932-96a7-4023-8a5e-407fbb8487c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:34.184004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e646932-96a7-4023-8a5e-407fbb8487c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:34.185845 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: WARNING py.warnings [None req-4e646932-96a7-4023-8a5e-407fbb8487c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:40:34.185845 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: warnings.warn(msg) Sep 26 05:40:34.185845 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: Sep 26 05:40:34.187053 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e646932-96a7-4023-8a5e-407fbb8487c6 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:34.231234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 754/1497] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:40:34 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:40:34.236195 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d6881-2924-46be-a4f9-49fce8830650 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:34.246365 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d6881-2924-46be-a4f9-49fce8830650 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:34.298796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:40:34.307584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07c543-aa9d-42dd-b426-685209d7792b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:34.308320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07c543-aa9d-42dd-b426-685209d7792b admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:34.308741 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07c543-aa9d-42dd-b426-685209d7792b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:34.309063 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07c543-aa9d-42dd-b426-685209d7792b admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:34.309779 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07c543-aa9d-42dd-b426-685209d7792b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:34.310942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: WARNING py.warnings [None req-ea07c543-aa9d-42dd-b426-685209d7792b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:40:34.310942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: warnings.warn(msg) Sep 26 05:40:34.310942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: Sep 26 05:40:34.312335 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07c543-aa9d-42dd-b426-685209d7792b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:34.338359 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 744/1498] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:40:34 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:40:34.345594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e646932-96a7-4023-8a5e-407fbb8487c6 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:34.358286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e646932-96a7-4023-8a5e-407fbb8487c6 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:34.364423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1af9ab-aa6f-4864-abb7-a7207bd0eb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:34.365358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1af9ab-aa6f-4864-abb7-a7207bd0eb26 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:34.365685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1af9ab-aa6f-4864-abb7-a7207bd0eb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:34.366078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1af9ab-aa6f-4864-abb7-a7207bd0eb26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:34.367027 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1af9ab-aa6f-4864-abb7-a7207bd0eb26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:34.368301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: WARNING py.warnings [None req-bd1af9ab-aa6f-4864-abb7-a7207bd0eb26 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:40:34.368301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: warnings.warn(msg) Sep 26 05:40:34.368301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: Sep 26 05:40:34.369564 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1af9ab-aa6f-4864-abb7-a7207bd0eb26 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:34.387107 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 755/1499] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:40:34.391499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07c543-aa9d-42dd-b426-685209d7792b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:34.401864 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07c543-aa9d-42dd-b426-685209d7792b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:34.406626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46f97b-b78e-4730-9545-947c92d413f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:34.407237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46f97b-b78e-4730-9545-947c92d413f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:34.407541 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46f97b-b78e-4730-9545-947c92d413f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:34.407777 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46f97b-b78e-4730-9545-947c92d413f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:34.408431 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46f97b-b78e-4730-9545-947c92d413f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:34.409480 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: WARNING py.warnings [None req-2e46f97b-b78e-4730-9545-947c92d413f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:40:34.409480 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: warnings.warn(msg) Sep 26 05:40:34.409480 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: Sep 26 05:40:34.410357 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46f97b-b78e-4730-9545-947c92d413f7 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:34.424865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 745/1500] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:40:34.429597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1af9ab-aa6f-4864-abb7-a7207bd0eb26 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:34.441081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1af9ab-aa6f-4864-abb7-a7207bd0eb26 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:34.447359 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f969e-2336-4352-8092-0c686944ba80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:34.448023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f969e-2336-4352-8092-0c686944ba80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:34.448294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f969e-2336-4352-8092-0c686944ba80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:34.448543 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f969e-2336-4352-8092-0c686944ba80 admin admin] PATH_INFO: `/v3/projects/e80ebd4320d94214963f5619a6465595/users/537debc8c0d34e80a52a2deca84a434d/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:34.465211 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f969e-2336-4352-8092-0c686944ba80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e80ebd4320d94214963f5619a6465595, user_id=537debc8c0d34e80a52a2deca84a434d, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:34.466241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: WARNING py.warnings [None req-ec4f969e-2336-4352-8092-0c686944ba80 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:40:34.466241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: warnings.warn(msg) Sep 26 05:40:34.466241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: Sep 26 05:40:34.468226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f969e-2336-4352-8092-0c686944ba80 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:34.484382 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 756/1501] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:40:34 2022] PUT /identity/v3/projects/e80ebd4320d94214963f5619a6465595/users/537debc8c0d34e80a52a2deca84a434d/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:40:34.497837 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ff2c2-17c8-49b5-b55d-4b51c4248f8f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:34.498137 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ff2c2-17c8-49b5-b55d-4b51c4248f8f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:34.498410 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ff2c2-17c8-49b5-b55d-4b51c4248f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:34.529775 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-640ff2c2-17c8-49b5-b55d-4b51c4248f8f None None] MFA Rules not processed for user `537debc8c0d34e80a52a2deca84a434d`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:40:34.531037 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-640ff2c2-17c8-49b5-b55d-4b51c4248f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:34.603505 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 746/1502] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:40:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:40:34.618208 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49c7b2-f62e-4ad3-9c4c-5a2d216394e9 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:34.618494 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49c7b2-f62e-4ad3-9c4c-5a2d216394e9 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:34.618781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49c7b2-f62e-4ad3-9c4c-5a2d216394e9 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:34.620134 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 757/1503] 10.222.0.57 () {54 vars in 963 bytes} [Mon Sep 26 05:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:40:34.625179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64741c09-91c9-4f29-a4df-99e9f6e1048e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:34.625479 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64741c09-91c9-4f29-a4df-99e9f6e1048e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:34.625752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64741c09-91c9-4f29-a4df-99e9f6e1048e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:34.812673 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-64741c09-91c9-4f29-a4df-99e9f6e1048e None None] MFA Rules not processed for user `46edd6aba77c4404977b1c39e8442b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:40:34.813970 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-64741c09-91c9-4f29-a4df-99e9f6e1048e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:34.902572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 747/1504] 10.222.0.57 () {58 vars in 1060 bytes} [Mon Sep 26 05:40:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:40:34.914248 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b49c7b2-f62e-4ad3-9c4c-5a2d216394e9 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:34.926498 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b49c7b2-f62e-4ad3-9c4c-5a2d216394e9 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:34.931406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084fc29-b102-4c49-ac79-03fa1c98fd9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:34.931629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084fc29-b102-4c49-ac79-03fa1c98fd9f service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:34.932054 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084fc29-b102-4c49-ac79-03fa1c98fd9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:34.932320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084fc29-b102-4c49-ac79-03fa1c98fd9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:34.942941 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084fc29-b102-4c49-ac79-03fa1c98fd9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:34.945714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084fc29-b102-4c49-ac79-03fa1c98fd9f service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:34.952036 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:40:34.964737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 758/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:40:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:40:35.014869 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecf4e9-a545-4e49-ad2f-b8dc900ebf52 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:35.015140 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecf4e9-a545-4e49-ad2f-b8dc900ebf52 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:35.015409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecf4e9-a545-4e49-ad2f-b8dc900ebf52 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:35.016711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 748/1506] 10.222.0.57 () {52 vars in 830 bytes} [Mon Sep 26 05:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:40:35.020457 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084fc29-b102-4c49-ac79-03fa1c98fd9f service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:35.086736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084fc29-b102-4c49-ac79-03fa1c98fd9f service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:35.091770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dab808-5bef-4173-b60a-1d1565ead669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:35.092412 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dab808-5bef-4173-b60a-1d1565ead669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:35.092677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dab808-5bef-4173-b60a-1d1565ead669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:35.092956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dab808-5bef-4173-b60a-1d1565ead669 admin admin] PATH_INFO: `/v3/users/537debc8c0d34e80a52a2deca84a434d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:35.094117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dab808-5bef-4173-b60a-1d1565ead669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=537debc8c0d34e80a52a2deca84a434d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:35.095124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: WARNING py.warnings [None req-82dab808-5bef-4173-b60a-1d1565ead669 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:40:35.095124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: warnings.warn(msg) Sep 26 05:40:35.095124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: Sep 26 05:40:35.096317 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dab808-5bef-4173-b60a-1d1565ead669 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:35.175456 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-82dab808-5bef-4173-b60a-1d1565ead669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '537debc8c0d34e80a52a2deca84a434d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.187673 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-82dab808-5bef-4173-b60a-1d1565ead669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '537debc8c0d34e80a52a2deca84a434d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.188556 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-82dab808-5bef-4173-b60a-1d1565ead669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '537debc8c0d34e80a52a2deca84a434d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.189590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-82dab808-5bef-4173-b60a-1d1565ead669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '537debc8c0d34e80a52a2deca84a434d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.206913 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-82dab808-5bef-4173-b60a-1d1565ead669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '537debc8c0d34e80a52a2deca84a434d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.240419 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 759/1507] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:40:35 2022] DELETE /identity/v3/users/537debc8c0d34e80a52a2deca84a434d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:40:35.245198 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13f166-a9d1-42c8-8664-c335bda258d1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:35.245478 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13f166-a9d1-42c8-8664-c335bda258d1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:35.245747 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13f166-a9d1-42c8-8664-c335bda258d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:35.265244 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8e13f166-a9d1-42c8-8664-c335bda258d1 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:40:35.266446 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8e13f166-a9d1-42c8-8664-c335bda258d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:35.276123 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:40:35.332730 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 749/1508] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:40:35.343121 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dab808-5bef-4173-b60a-1d1565ead669 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:35.344704 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-82dab808-5bef-4173-b60a-1d1565ead669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:35.425842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dab808-5bef-4173-b60a-1d1565ead669 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:35.429815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c32849-9ee5-499e-8dad-5efc3bca911e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:35.430431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c32849-9ee5-499e-8dad-5efc3bca911e service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:35.430723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c32849-9ee5-499e-8dad-5efc3bca911e service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:35.430995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c32849-9ee5-499e-8dad-5efc3bca911e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:35.440849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c32849-9ee5-499e-8dad-5efc3bca911e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:35.442332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c32849-9ee5-499e-8dad-5efc3bca911e service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:35.447616 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:40:35.453241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 760/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:40:35.840956 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13f166-a9d1-42c8-8664-c335bda258d1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:40:35.843706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8e13f166-a9d1-42c8-8664-c335bda258d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:40:35.858202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13f166-a9d1-42c8-8664-c335bda258d1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:40:35.863721 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f958b-2959-4010-8497-1bab54d13fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:40:35.864384 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f958b-2959-4010-8497-1bab54d13fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:40:35.864696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f958b-2959-4010-8497-1bab54d13fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:40:35.865036 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f958b-2959-4010-8497-1bab54d13fa6 admin admin] PATH_INFO: `/v3/projects/e80ebd4320d94214963f5619a6465595` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:40:35.866479 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f958b-2959-4010-8497-1bab54d13fa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e80ebd4320d94214963f5619a6465595)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:40:35.867535 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: WARNING py.warnings [None req-984f958b-2959-4010-8497-1bab54d13fa6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:40:35.867535 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: warnings.warn(msg) Sep 26 05:40:35.867535 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: Sep 26 05:40:35.868723 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f958b-2959-4010-8497-1bab54d13fa6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:40:35.937092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-984f958b-2959-4010-8497-1bab54d13fa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e80ebd4320d94214963f5619a6465595', 'request_id': 'req-984f958b-2959-4010-8497-1bab54d13fa6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.952810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-984f958b-2959-4010-8497-1bab54d13fa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e80ebd4320d94214963f5619a6465595', 'request_id': 'req-984f958b-2959-4010-8497-1bab54d13fa6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.967891 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-984f958b-2959-4010-8497-1bab54d13fa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e80ebd4320d94214963f5619a6465595', 'request_id': 'req-984f958b-2959-4010-8497-1bab54d13fa6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.981358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-984f958b-2959-4010-8497-1bab54d13fa6 admin admin] The token cache is being invalidate because project e80ebd4320d94214963f5619a6465595 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:40:35.981664 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-984f958b-2959-4010-8497-1bab54d13fa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.982504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-984f958b-2959-4010-8497-1bab54d13fa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:40:35.984160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 750/1510] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:40:35 2022] DELETE /identity/v3/projects/e80ebd4320d94214963f5619a6465595 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:40:45.977586 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:42:07.561490 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165f822-a870-4a82-87c3-af41c1d5bd5a None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:07.562372 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165f822-a870-4a82-87c3-af41c1d5bd5a None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:07.562372 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165f822-a870-4a82-87c3-af41c1d5bd5a None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:07.564378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:42:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:42:07.569467 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9784bef-8ce5-4ca5-adcc-f4759beb5d87 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:07.569959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9784bef-8ce5-4ca5-adcc-f4759beb5d87 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:07.569959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9784bef-8ce5-4ca5-adcc-f4759beb5d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:07.592537 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f9784bef-8ce5-4ca5-adcc-f4759beb5d87 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:42:07.593794 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f9784bef-8ce5-4ca5-adcc-f4759beb5d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:07.661692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:42:07.813475 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165f822-a870-4a82-87c3-af41c1d5bd5a None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:42:07.816634 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-3165f822-a870-4a82-87c3-af41c1d5bd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:07.937035 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165f822-a870-4a82-87c3-af41c1d5bd5a None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:42:07.943021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01681bb6-398b-4fd7-9ba1-986bcfded750 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:42:07.943631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01681bb6-398b-4fd7-9ba1-986bcfded750 service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:07.943903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01681bb6-398b-4fd7-9ba1-986bcfded750 service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:07.944171 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01681bb6-398b-4fd7-9ba1-986bcfded750 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:07.952651 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01681bb6-398b-4fd7-9ba1-986bcfded750 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:42:07.954251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01681bb6-398b-4fd7-9ba1-986bcfded750 service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:42:07.964405 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 762/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:42:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:42:09.049564 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755346ea-cce5-4cde-8d66-6435336d466b None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:09.050174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755346ea-cce5-4cde-8d66-6435336d466b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:09.050174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755346ea-cce5-4cde-8d66-6435336d466b None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:09.052021 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:42:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:42:09.058604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7cecb-6245-4f72-a2be-294680ef8d4b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:09.059098 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7cecb-6245-4f72-a2be-294680ef8d4b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:09.059098 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7cecb-6245-4f72-a2be-294680ef8d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:09.080556 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-12e7cecb-6245-4f72-a2be-294680ef8d4b None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:42:09.081842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-12e7cecb-6245-4f72-a2be-294680ef8d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:09.091798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:42:09.174187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162b097-7611-458e-94a0-988ef19f90d0 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:09.174476 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162b097-7611-458e-94a0-988ef19f90d0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:09.174744 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162b097-7611-458e-94a0-988ef19f90d0 None None] PATH_INFO: `/v3` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:09.176079 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 753/1516] 10.222.0.25 () {54 vars in 925 bytes} [Mon Sep 26 05:42:09 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:42:09.183416 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2748c-3d37-4a3f-91ce-7281b602fc7f None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:09.183710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2748c-3d37-4a3f-91ce-7281b602fc7f None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:09.183981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2748c-3d37-4a3f-91ce-7281b602fc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:09.241737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-7fc2748c-3d37-4a3f-91ce-7281b602fc7f None None] MFA Rules not processed for user `67fcd65eeb994b2381b83480e030f2ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:42:09.242950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7fc2748c-3d37-4a3f-91ce-7281b602fc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:09.245872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162b097-7611-458e-94a0-988ef19f90d0 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:42:09.260382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162b097-7611-458e-94a0-988ef19f90d0 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:42:09.266088 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83e256-956e-4b90-9787-6ad75fd2a4c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:42:09.266685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83e256-956e-4b90-9787-6ad75fd2a4c5 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:09.266960 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83e256-956e-4b90-9787-6ad75fd2a4c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:09.267224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83e256-956e-4b90-9787-6ad75fd2a4c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:09.275737 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83e256-956e-4b90-9787-6ad75fd2a4c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:42:09.277385 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83e256-956e-4b90-9787-6ad75fd2a4c5 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:42:09.287213 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 754/1517] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:42:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:42:09.317490 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 764/1518] 10.222.0.25 () {58 vars in 1015 bytes} [Mon Sep 26 05:42:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:42:17.989684 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd582f-80ed-4db3-83b0-f90ce3ee2053 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:17.990230 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd582f-80ed-4db3-83b0-f90ce3ee2053 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:17.990230 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd582f-80ed-4db3-83b0-f90ce3ee2053 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:17.991664 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 755/1519] 10.222.0.57 () {54 vars in 961 bytes} [Mon Sep 26 05:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:42:17.997280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666825b1-5922-453e-8080-3112742d1516 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:17.997733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666825b1-5922-453e-8080-3112742d1516 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:17.997839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666825b1-5922-453e-8080-3112742d1516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:18.019229 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-666825b1-5922-453e-8080-3112742d1516 None None] MFA Rules not processed for user `67fcd65eeb994b2381b83480e030f2ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:42:18.020493 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-666825b1-5922-453e-8080-3112742d1516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:18.030073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 765/1520] 10.222.0.57 () {58 vars in 1058 bytes} [Mon Sep 26 05:42:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:42:18.037966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd582f-80ed-4db3-83b0-f90ce3ee2053 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:42:18.053157 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd582f-80ed-4db3-83b0-f90ce3ee2053 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:42:18.058825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5de66-d388-4c66-bb4b-a374d6d27314 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:42:18.059470 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5de66-d388-4c66-bb4b-a374d6d27314 service placement] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:18.059741 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5de66-d388-4c66-bb4b-a374d6d27314 service placement] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:18.060008 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5de66-d388-4c66-bb4b-a374d6d27314 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:18.069756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5de66-d388-4c66-bb4b-a374d6d27314 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:42:18.071372 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5de66-d388-4c66-bb4b-a374d6d27314 service placement] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:42:18.082045 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 756/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:42:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:42:20.803489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f481b-f597-48fe-bbe6-8953443994d5 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:20.804112 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f481b-f597-48fe-bbe6-8953443994d5 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:20.804112 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f481b-f597-48fe-bbe6-8953443994d5 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:20.805439 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:42:20.812826 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43775d9c-d89b-4858-be4f-cca31beed262 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:20.813346 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43775d9c-d89b-4858-be4f-cca31beed262 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:20.813454 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43775d9c-d89b-4858-be4f-cca31beed262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:20.843911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-43775d9c-d89b-4858-be4f-cca31beed262 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:42:20.845128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-43775d9c-d89b-4858-be4f-cca31beed262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:20.854446 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:42:21.012186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f481b-f597-48fe-bbe6-8953443994d5 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:42:21.027256 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f481b-f597-48fe-bbe6-8953443994d5 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:42:21.032921 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294f1f1-2130-4da4-98a3-408c08a7676a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:42:21.033548 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294f1f1-2130-4da4-98a3-408c08a7676a service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:21.033829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294f1f1-2130-4da4-98a3-408c08a7676a service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:21.034107 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294f1f1-2130-4da4-98a3-408c08a7676a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:21.040302 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:42:21.042751 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294f1f1-2130-4da4-98a3-408c08a7676a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:42:21.044695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294f1f1-2130-4da4-98a3-408c08a7676a service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:42:21.054494 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 767/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:42:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:42:22.122427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1b0d2-a39f-4429-b3de-b0c65e924761 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:22.123067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1b0d2-a39f-4429-b3de-b0c65e924761 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:22.123067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1b0d2-a39f-4429-b3de-b0c65e924761 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:22.124325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 05:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:42:22.132047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951edf1a-f83a-44b9-874a-5d2432071d27 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:22.132820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951edf1a-f83a-44b9-874a-5d2432071d27 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:22.132820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951edf1a-f83a-44b9-874a-5d2432071d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:22.159766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-951edf1a-f83a-44b9-874a-5d2432071d27 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:42:22.160981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-951edf1a-f83a-44b9-874a-5d2432071d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:22.169965 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 05:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:42:22.329373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1b0d2-a39f-4429-b3de-b0c65e924761 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:42:22.349852 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1b0d2-a39f-4429-b3de-b0c65e924761 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:42:22.355535 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f91dcf-1dd0-4de1-9efe-c13212a3df2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:42:22.356139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f91dcf-1dd0-4de1-9efe-c13212a3df2d service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:22.356418 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f91dcf-1dd0-4de1-9efe-c13212a3df2d service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:22.356689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f91dcf-1dd0-4de1-9efe-c13212a3df2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:22.365031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f91dcf-1dd0-4de1-9efe-c13212a3df2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:42:22.366662 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f91dcf-1dd0-4de1-9efe-c13212a3df2d service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:42:22.376613 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 759/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:42:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:42:26.466250 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519a079-f77c-43b1-a263-238b5da8b77f None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:26.466866 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519a079-f77c-43b1-a263-238b5da8b77f None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:26.466866 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519a079-f77c-43b1-a263-238b5da8b77f None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:26.468169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:42:26.474553 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b29dd-bc1b-4c26-9d85-717e15368288 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:26.475291 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b29dd-bc1b-4c26-9d85-717e15368288 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:26.475487 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b29dd-bc1b-4c26-9d85-717e15368288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:26.498321 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f46b29dd-bc1b-4c26-9d85-717e15368288 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:42:26.499466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f46b29dd-bc1b-4c26-9d85-717e15368288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:26.508458 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:42:26.798245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519a079-f77c-43b1-a263-238b5da8b77f None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:42:26.800374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-e519a079-f77c-43b1-a263-238b5da8b77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:26.861777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519a079-f77c-43b1-a263-238b5da8b77f None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:42:26.865785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189847aa-c361-477c-bedf-0ba217677505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:42:26.866403 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189847aa-c361-477c-bedf-0ba217677505 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:26.866701 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189847aa-c361-477c-bedf-0ba217677505 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:26.866960 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189847aa-c361-477c-bedf-0ba217677505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:26.876780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189847aa-c361-477c-bedf-0ba217677505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:42:26.878248 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189847aa-c361-477c-bedf-0ba217677505 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:42:26.887813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 770/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:42:29.027292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7414b5f-e8f3-42fe-a61d-5c3e34b78122 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:29.027954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7414b5f-e8f3-42fe-a61d-5c3e34b78122 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:29.027954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7414b5f-e8f3-42fe-a61d-5c3e34b78122 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:29.029292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:42:29.035600 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97052c-988a-48dc-8d3c-91b6dad3de36 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:29.036357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97052c-988a-48dc-8d3c-91b6dad3de36 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:29.036511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97052c-988a-48dc-8d3c-91b6dad3de36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:29.065780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-fa97052c-988a-48dc-8d3c-91b6dad3de36 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:42:29.066996 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-fa97052c-988a-48dc-8d3c-91b6dad3de36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:29.076698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:42:29.366487 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7414b5f-e8f3-42fe-a61d-5c3e34b78122 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:42:29.384818 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7414b5f-e8f3-42fe-a61d-5c3e34b78122 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:42:29.390746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d5d2db-090b-457a-b8b6-8e7b5e758bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:42:29.391353 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5d2db-090b-457a-b8b6-8e7b5e758bae service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:29.391631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5d2db-090b-457a-b8b6-8e7b5e758bae service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:29.391908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5d2db-090b-457a-b8b6-8e7b5e758bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:29.401364 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d5d2db-090b-457a-b8b6-8e7b5e758bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:42:29.402814 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d5d2db-090b-457a-b8b6-8e7b5e758bae service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:42:29.412653 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 762/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:42:33.065737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae489b-c433-4df0-bf7c-393a0340ebe8 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:33.066344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae489b-c433-4df0-bf7c-393a0340ebe8 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:33.066344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae489b-c433-4df0-bf7c-393a0340ebe8 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:33.067617 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:42:33.074680 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d193a-1085-4e17-b944-4cb9c900270c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:33.075310 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d193a-1085-4e17-b944-4cb9c900270c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:33.075310 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d193a-1085-4e17-b944-4cb9c900270c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:33.097953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f07d193a-1085-4e17-b944-4cb9c900270c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:42:33.099226 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f07d193a-1085-4e17-b944-4cb9c900270c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:33.109531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:42:33.412008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae489b-c433-4df0-bf7c-393a0340ebe8 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:42:33.413940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-27ae489b-c433-4df0-bf7c-393a0340ebe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:33.474078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae489b-c433-4df0-bf7c-393a0340ebe8 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:42:33.478272 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14d0b6-57d6-44da-8b4d-75ee5d806b5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:42:33.478879 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14d0b6-57d6-44da-8b4d-75ee5d806b5f service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:33.479157 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14d0b6-57d6-44da-8b4d-75ee5d806b5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:33.479427 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14d0b6-57d6-44da-8b4d-75ee5d806b5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:33.489320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb14d0b6-57d6-44da-8b4d-75ee5d806b5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:42:33.490762 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb14d0b6-57d6-44da-8b4d-75ee5d806b5f service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:42:33.501019 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 773/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:42:35.692074 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6493e-1707-4531-867e-ad35833a6527 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:35.692687 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6493e-1707-4531-867e-ad35833a6527 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:35.692687 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6493e-1707-4531-867e-ad35833a6527 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:35.694068 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 05:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:42:35.699681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0520955-9f6e-4b44-b362-b6f5fc9d7fd3 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:35.700152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0520955-9f6e-4b44-b362-b6f5fc9d7fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:35.700251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0520955-9f6e-4b44-b362-b6f5fc9d7fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:35.720644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d0520955-9f6e-4b44-b362-b6f5fc9d7fd3 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:42:35.721870 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d0520955-9f6e-4b44-b362-b6f5fc9d7fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:35.730927 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 05:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:42:36.029348 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6493e-1707-4531-867e-ad35833a6527 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:42:36.031202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-dfb6493e-1707-4531-867e-ad35833a6527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:42:36.039129 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=61 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:42:36.045391 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6493e-1707-4531-867e-ad35833a6527 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:42:36.049319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003a2da-1f4b-4ae4-b4e8-0b89ff2d141f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:42:36.049942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003a2da-1f4b-4ae4-b4e8-0b89ff2d141f service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:42:36.050219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003a2da-1f4b-4ae4-b4e8-0b89ff2d141f service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:42:36.050489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003a2da-1f4b-4ae4-b4e8-0b89ff2d141f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:42:36.060501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003a2da-1f4b-4ae4-b4e8-0b89ff2d141f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:42:36.061965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003a2da-1f4b-4ae4-b4e8-0b89ff2d141f service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:42:36.071875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 765/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:42:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:42:45.715516 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=63 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:42:46.067432 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:01.270725 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaac9ef-13e5-47d0-83b9-ca5ffaed1dbd None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.271939 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaac9ef-13e5-47d0-83b9-ca5ffaed1dbd None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.272435 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaac9ef-13e5-47d0-83b9-ca5ffaed1dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.306530 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-5aaac9ef-13e5-47d0-83b9-ca5ffaed1dbd None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:01.308600 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5aaac9ef-13e5-47d0-83b9-ca5ffaed1dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:01.317577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 775/1540] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:01.327191 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a4cf8-7b0e-4dc4-847e-322ccddf83ed None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.328528 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a4cf8-7b0e-4dc4-847e-322ccddf83ed None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.329019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a4cf8-7b0e-4dc4-847e-322ccddf83ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.357951 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-646a4cf8-7b0e-4dc4-847e-322ccddf83ed None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:01.359852 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-646a4cf8-7b0e-4dc4-847e-322ccddf83ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:01.373072 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 766/1541] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:01.378219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaac9ef-13e5-47d0-83b9-ca5ffaed1dbd None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:01.395612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaac9ef-13e5-47d0-83b9-ca5ffaed1dbd None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:01.408245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531cf5c-8bba-4485-87d4-b5099c1f4a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:01.409329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531cf5c-8bba-4485-87d4-b5099c1f4a7a admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.410083 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531cf5c-8bba-4485-87d4-b5099c1f4a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.410570 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531cf5c-8bba-4485-87d4-b5099c1f4a7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.411494 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531cf5c-8bba-4485-87d4-b5099c1f4a7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:44:01.412015 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531cf5c-8bba-4485-87d4-b5099c1f4a7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:01.413301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: WARNING py.warnings [None req-8531cf5c-8bba-4485-87d4-b5099c1f4a7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:44:01.413301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: warnings.warn(msg) Sep 26 05:44:01.413301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: Sep 26 05:44:01.414655 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531cf5c-8bba-4485-87d4-b5099c1f4a7a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:01.433085 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 776/1542] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:44:01.436973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cf2be-b5e5-4afe-a80d-754565a877ec None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.437548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cf2be-b5e5-4afe-a80d-754565a877ec None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.438051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cf2be-b5e5-4afe-a80d-754565a877ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.444092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531cf5c-8bba-4485-87d4-b5099c1f4a7a admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:01.469439 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531cf5c-8bba-4485-87d4-b5099c1f4a7a admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:01.474764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-3e8cf2be-b5e5-4afe-a80d-754565a877ec None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:01.476647 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3e8cf2be-b5e5-4afe-a80d-754565a877ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:01.484209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20dd5c-db5c-4cd9-ab98-60520e1e0517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:01.485216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20dd5c-db5c-4cd9-ab98-60520e1e0517 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.485726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20dd5c-db5c-4cd9-ab98-60520e1e0517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.486234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20dd5c-db5c-4cd9-ab98-60520e1e0517 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.488194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20dd5c-db5c-4cd9-ab98-60520e1e0517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:01.490444 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20dd5c-db5c-4cd9-ab98-60520e1e0517 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:01.497476 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 767/1543] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:01.523145 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf3e2d-487b-4428-a260-529caaf0f451 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.523867 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf3e2d-487b-4428-a260-529caaf0f451 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.524346 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf3e2d-487b-4428-a260-529caaf0f451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.557874 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 777/1544] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:44:01 2022] POST /identity/v3/projects => generated 391 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:01.563237 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20dd5c-db5c-4cd9-ab98-60520e1e0517 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:01.570774 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-23bf3e2d-487b-4428-a260-529caaf0f451 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:01.576833 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-23bf3e2d-487b-4428-a260-529caaf0f451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:01.679806 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 768/1545] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:44:01.687155 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4827e8-26c0-451c-95db-0138dda0678a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.687548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4827e8-26c0-451c-95db-0138dda0678a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.687737 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4827e8-26c0-451c-95db-0138dda0678a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.718963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ed4827e8-26c0-451c-95db-0138dda0678a None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:01.719216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20dd5c-db5c-4cd9-ab98-60520e1e0517 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:01.720261 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ed4827e8-26c0-451c-95db-0138dda0678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:01.733531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078bc73-ad31-4aea-a1d9-bfe993f38daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:01.734069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 769/1546] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:44:01.735150 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4827e8-26c0-451c-95db-0138dda0678a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:01.735520 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078bc73-ad31-4aea-a1d9-bfe993f38daa admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.736540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078bc73-ad31-4aea-a1d9-bfe993f38daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.737259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078bc73-ad31-4aea-a1d9-bfe993f38daa admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.738515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078bc73-ad31-4aea-a1d9-bfe993f38daa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:01.745870 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4827e8-26c0-451c-95db-0138dda0678a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:01.747373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: WARNING py.warnings [None req-d078bc73-ad31-4aea-a1d9-bfe993f38daa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:44:01.747373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: warnings.warn(msg) Sep 26 05:44:01.747373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: Sep 26 05:44:01.748734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078bc73-ad31-4aea-a1d9-bfe993f38daa admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:01.751983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af6005-b190-453c-8ed3-3183d22d66e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:01.752611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af6005-b190-453c-8ed3-3183d22d66e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.752888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af6005-b190-453c-8ed3-3183d22d66e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.753182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af6005-b190-453c-8ed3-3183d22d66e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.753838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af6005-b190-453c-8ed3-3183d22d66e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:44:01.754138 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af6005-b190-453c-8ed3-3183d22d66e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:01.755804 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af6005-b190-453c-8ed3-3183d22d66e5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:01.774002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 770/1547] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:44:01.775589 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3def6a-7f5a-46b9-bb84-a8f7fc9b3bbf None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.775932 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3def6a-7f5a-46b9-bb84-a8f7fc9b3bbf None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.776176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 778/1548] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:44:01 2022] POST /identity/v3/users => generated 296 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:01.776310 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3def6a-7f5a-46b9-bb84-a8f7fc9b3bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.777608 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199294d-c44e-485d-af4c-23606cc3737b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.778039 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199294d-c44e-485d-af4c-23606cc3737b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.778360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199294d-c44e-485d-af4c-23606cc3737b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.798036 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ae3def6a-7f5a-46b9-bb84-a8f7fc9b3bbf None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:01.799260 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ae3def6a-7f5a-46b9-bb84-a8f7fc9b3bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:01.801764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-1199294d-c44e-485d-af4c-23606cc3737b None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:01.802943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-1199294d-c44e-485d-af4c-23606cc3737b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:01.807950 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 771/1549] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:01.816153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 779/1550] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:01.816153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68234c71-f00b-462b-95b4-9a84acf08276 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.816153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68234c71-f00b-462b-95b4-9a84acf08276 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.816153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68234c71-f00b-462b-95b4-9a84acf08276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.816998 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1634a7-0da5-4c62-ac02-9f801057c974 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.816998 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1634a7-0da5-4c62-ac02-9f801057c974 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.816998 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1634a7-0da5-4c62-ac02-9f801057c974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.839462 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-1c1634a7-0da5-4c62-ac02-9f801057c974 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:01.840668 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1c1634a7-0da5-4c62-ac02-9f801057c974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:01.840851 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-68234c71-f00b-462b-95b4-9a84acf08276 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:01.842114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-68234c71-f00b-462b-95b4-9a84acf08276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:01.849474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 772/1551] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:01.850333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1634a7-0da5-4c62-ac02-9f801057c974 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:01.851257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 780/1552] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:01.852038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68234c71-f00b-462b-95b4-9a84acf08276 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:01.862204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68234c71-f00b-462b-95b4-9a84acf08276 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:01.867148 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa75db-f6bd-49a7-9bd3-5091b3e2a963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:01.867842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa75db-f6bd-49a7-9bd3-5091b3e2a963 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.867967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1634a7-0da5-4c62-ac02-9f801057c974 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:01.868130 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa75db-f6bd-49a7-9bd3-5091b3e2a963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.868535 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa75db-f6bd-49a7-9bd3-5091b3e2a963 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.869287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa75db-f6bd-49a7-9bd3-5091b3e2a963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:01.871268 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa75db-f6bd-49a7-9bd3-5091b3e2a963 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:01.872981 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa4ae8-de8d-48c0-9327-c483920cb2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:01.873701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa4ae8-de8d-48c0-9327-c483920cb2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.874050 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa4ae8-de8d-48c0-9327-c483920cb2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.874388 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa4ae8-de8d-48c0-9327-c483920cb2b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.875041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa4ae8-de8d-48c0-9327-c483920cb2b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:01.876630 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa4ae8-de8d-48c0-9327-c483920cb2b1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:01.888754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 773/1553] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:01.891723 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4e2e6-1554-42c0-a2ab-49ef710ddbc5 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.892613 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4e2e6-1554-42c0-a2ab-49ef710ddbc5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.893447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4e2e6-1554-42c0-a2ab-49ef710ddbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.905823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 781/1554] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:44:01 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:01.907158 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c789f-c001-46d5-a9f5-68362130edbc None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.907538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c789f-c001-46d5-a9f5-68362130edbc None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.907900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c789f-c001-46d5-a9f5-68362130edbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.915075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-62f4e2e6-1554-42c0-a2ab-49ef710ddbc5 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:01.916290 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-62f4e2e6-1554-42c0-a2ab-49ef710ddbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:01.928113 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d10c789f-c001-46d5-a9f5-68362130edbc None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:01.929350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d10c789f-c001-46d5-a9f5-68362130edbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:01.980500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 774/1555] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:01.981786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bff7d-5d11-4331-9d1e-219e1582d7f7 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:01.982069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bff7d-5d11-4331-9d1e-219e1582d7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:01.982341 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bff7d-5d11-4331-9d1e-219e1582d7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:01.992863 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 782/1556] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:01.993936 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c789f-c001-46d5-a9f5-68362130edbc None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.002234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-898bff7d-5d11-4331-9d1e-219e1582d7f7 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:02.003453 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-898bff7d-5d11-4331-9d1e-219e1582d7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:02.003988 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c789f-c001-46d5-a9f5-68362130edbc None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.009000 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26d066-99e4-4c92-83e3-341eccd25a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.009651 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26d066-99e4-4c92-83e3-341eccd25a26 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.009929 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26d066-99e4-4c92-83e3-341eccd25a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.010190 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26d066-99e4-4c92-83e3-341eccd25a26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.010863 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26d066-99e4-4c92-83e3-341eccd25a26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:44:02.011156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26d066-99e4-4c92-83e3-341eccd25a26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.011812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 775/1557] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:02.016087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26d066-99e4-4c92-83e3-341eccd25a26 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.016218 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35941fd-c2a4-4f48-9f53-9a784858736b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.016218 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35941fd-c2a4-4f48-9f53-9a784858736b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.016218 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35941fd-c2a4-4f48-9f53-9a784858736b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.024139 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:02.029342 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 783/1558] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:44:02.030645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26d066-99e4-4c92-83e3-341eccd25a26 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.034841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b35941fd-c2a4-4f48-9f53-9a784858736b None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:02.036154 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b35941fd-c2a4-4f48-9f53-9a784858736b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:02.043166 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26d066-99e4-4c92-83e3-341eccd25a26 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.044422 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 776/1559] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:02.045827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f993d-58d8-4327-8b5f-e6a2c4a77bb9 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.046049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f993d-58d8-4327-8b5f-e6a2c4a77bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.046322 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f993d-58d8-4327-8b5f-e6a2c4a77bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.048915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c0da4-0504-4222-b57a-181cbfe33199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.049574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c0da4-0504-4222-b57a-181cbfe33199 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.049852 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c0da4-0504-4222-b57a-181cbfe33199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.050118 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c0da4-0504-4222-b57a-181cbfe33199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.050759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c0da4-0504-4222-b57a-181cbfe33199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.052370 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c0da4-0504-4222-b57a-181cbfe33199 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.067336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 784/1560] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:02.067336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c0da4-0504-4222-b57a-181cbfe33199 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.068815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-691f993d-58d8-4327-8b5f-e6a2c4a77bb9 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:02.070301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-691f993d-58d8-4327-8b5f-e6a2c4a77bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:02.075718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c0da4-0504-4222-b57a-181cbfe33199 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.078730 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 777/1561] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:02.081303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c87622-7841-497a-a581-b18a48314d71 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.081303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c87622-7841-497a-a581-b18a48314d71 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.081554 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48205d6-4ee5-424a-b483-2559cb6581af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.081836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c87622-7841-497a-a581-b18a48314d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.081926 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48205d6-4ee5-424a-b483-2559cb6581af admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.081926 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48205d6-4ee5-424a-b483-2559cb6581af admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.081926 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48205d6-4ee5-424a-b483-2559cb6581af admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.082408 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48205d6-4ee5-424a-b483-2559cb6581af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.084392 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48205d6-4ee5-424a-b483-2559cb6581af admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.106572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b1c87622-7841-497a-a581-b18a48314d71 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:02.108040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b1c87622-7841-497a-a581-b18a48314d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:02.109862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 785/1562] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:44:02 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:02.110479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48205d6-4ee5-424a-b483-2559cb6581af admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.120767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 778/1563] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:02.124997 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48205d6-4ee5-424a-b483-2559cb6581af admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.125124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c87622-7841-497a-a581-b18a48314d71 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.125417 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bed6f1-cd9f-411d-b9bb-3f2d85f5de03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.126060 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bed6f1-cd9f-411d-b9bb-3f2d85f5de03 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.126316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bed6f1-cd9f-411d-b9bb-3f2d85f5de03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.126587 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bed6f1-cd9f-411d-b9bb-3f2d85f5de03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.127240 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bed6f1-cd9f-411d-b9bb-3f2d85f5de03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:44:02.128004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bed6f1-cd9f-411d-b9bb-3f2d85f5de03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.129241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bed6f1-cd9f-411d-b9bb-3f2d85f5de03 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.131955 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c87622-7841-497a-a581-b18a48314d71 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.137166 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04708510-0de7-4850-b66f-642c2aa192fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.137821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04708510-0de7-4850-b66f-642c2aa192fc admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.138097 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04708510-0de7-4850-b66f-642c2aa192fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.138394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04708510-0de7-4850-b66f-642c2aa192fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.139034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04708510-0de7-4850-b66f-642c2aa192fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:44:02.139332 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04708510-0de7-4850-b66f-642c2aa192fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.140953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04708510-0de7-4850-b66f-642c2aa192fc admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.143925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 786/1564] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:44:02.144835 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bed6f1-cd9f-411d-b9bb-3f2d85f5de03 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.155839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bed6f1-cd9f-411d-b9bb-3f2d85f5de03 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.156944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 779/1565] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:44:02.157916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04708510-0de7-4850-b66f-642c2aa192fc admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.160793 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9eb971-a9ff-4478-aa94-881a30b2bccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.161486 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9eb971-a9ff-4478-aa94-881a30b2bccc admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.161766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9eb971-a9ff-4478-aa94-881a30b2bccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.162231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9eb971-a9ff-4478-aa94-881a30b2bccc admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.162869 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9eb971-a9ff-4478-aa94-881a30b2bccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.164763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:02.164978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9eb971-a9ff-4478-aa94-881a30b2bccc admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.169260 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04708510-0de7-4850-b66f-642c2aa192fc admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.174598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc8627-d7fa-41b1-9128-40aa5904a913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.175207 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc8627-d7fa-41b1-9128-40aa5904a913 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.175484 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc8627-d7fa-41b1-9128-40aa5904a913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.175751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc8627-d7fa-41b1-9128-40aa5904a913 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.176398 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc8627-d7fa-41b1-9128-40aa5904a913 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:44:02.176757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc8627-d7fa-41b1-9128-40aa5904a913 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.178391 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc8627-d7fa-41b1-9128-40aa5904a913 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.192176 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 780/1566] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:44:02.192984 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc8627-d7fa-41b1-9128-40aa5904a913 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.202766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 787/1567] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:44:02 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:02.204121 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1e8f5-745c-4eb9-a11c-253fbef87912 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.204411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1e8f5-745c-4eb9-a11c-253fbef87912 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.204530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc8627-d7fa-41b1-9128-40aa5904a913 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.204712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1e8f5-745c-4eb9-a11c-253fbef87912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.225331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-8fe1e8f5-745c-4eb9-a11c-253fbef87912 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:02.226588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-8fe1e8f5-745c-4eb9-a11c-253fbef87912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:02.269073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33faa44-e2ca-416c-a04c-4d772e656852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.269767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33faa44-e2ca-416c-a04c-4d772e656852 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.270107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33faa44-e2ca-416c-a04c-4d772e656852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.270377 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33faa44-e2ca-416c-a04c-4d772e656852 admin admin] PATH_INFO: `/v3/projects/c95cba08e61d4d199a3c69d99c5bc77b/users/342ce12282264d40b2f2bb6f1fc6ba25/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.286504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33faa44-e2ca-416c-a04c-4d772e656852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95cba08e61d4d199a3c69d99c5bc77b, user_id=342ce12282264d40b2f2bb6f1fc6ba25, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.287551 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: WARNING py.warnings [None req-c33faa44-e2ca-416c-a04c-4d772e656852 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:44:02.287551 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: warnings.warn(msg) Sep 26 05:44:02.287551 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: Sep 26 05:44:02.289490 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33faa44-e2ca-416c-a04c-4d772e656852 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.293768 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 788/1568] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:02.295077 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1e8f5-745c-4eb9-a11c-253fbef87912 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.304097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1e8f5-745c-4eb9-a11c-253fbef87912 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.305505 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 781/1569] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:02 2022] PUT /identity/v3/projects/c95cba08e61d4d199a3c69d99c5bc77b/users/342ce12282264d40b2f2bb6f1fc6ba25/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:02.310187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33faa44-e2ca-416c-a04c-4d772e656852 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.365208 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b06e4-29c7-4a33-91e9-5514e56f477c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.365887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b06e4-29c7-4a33-91e9-5514e56f477c admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.366105 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b06e4-29c7-4a33-91e9-5514e56f477c admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.366375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b06e4-29c7-4a33-91e9-5514e56f477c admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.367021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b06e4-29c7-4a33-91e9-5514e56f477c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.368618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b06e4-29c7-4a33-91e9-5514e56f477c admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.378944 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 789/1570] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:02.382276 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33faa44-e2ca-416c-a04c-4d772e656852 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.382759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b06e4-29c7-4a33-91e9-5514e56f477c admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.386282 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0b3c9-f3f4-4af7-85a4-805c23f27c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.386902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0b3c9-f3f4-4af7-85a4-805c23f27c3b admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.388756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0b3c9-f3f4-4af7-85a4-805c23f27c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.388756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0b3c9-f3f4-4af7-85a4-805c23f27c3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.388756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0b3c9-f3f4-4af7-85a4-805c23f27c3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:44:02.388756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0b3c9-f3f4-4af7-85a4-805c23f27c3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.393683 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0b3c9-f3f4-4af7-85a4-805c23f27c3b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.396507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b06e4-29c7-4a33-91e9-5514e56f477c admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.401728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542a9cb-4f73-4915-9e09-1c2671a84124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.402120 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 782/1571] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:44:02.402337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542a9cb-4f73-4915-9e09-1c2671a84124 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.402777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542a9cb-4f73-4915-9e09-1c2671a84124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.402981 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0b3c9-f3f4-4af7-85a4-805c23f27c3b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.403087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542a9cb-4f73-4915-9e09-1c2671a84124 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.403652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542a9cb-4f73-4915-9e09-1c2671a84124 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.405656 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542a9cb-4f73-4915-9e09-1c2671a84124 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.420955 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0b3c9-f3f4-4af7-85a4-805c23f27c3b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.425738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6cd99a-f603-47e0-9e3a-c99ce585a5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.426353 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cd99a-f603-47e0-9e3a-c99ce585a5be admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.426714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cd99a-f603-47e0-9e3a-c99ce585a5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.426995 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cd99a-f603-47e0-9e3a-c99ce585a5be admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.427634 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6cd99a-f603-47e0-9e3a-c99ce585a5be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.429079 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: WARNING py.warnings [None req-8d6cd99a-f603-47e0-9e3a-c99ce585a5be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:44:02.429079 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: warnings.warn(msg) Sep 26 05:44:02.429079 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: Sep 26 05:44:02.429963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6cd99a-f603-47e0-9e3a-c99ce585a5be admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.441636 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 790/1572] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:44:02 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:02.445233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542a9cb-4f73-4915-9e09-1c2671a84124 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.459388 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:02.460093 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 783/1573] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:44:02 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:02.463891 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6cd99a-f603-47e0-9e3a-c99ce585a5be admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.517036 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:02.518230 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542a9cb-4f73-4915-9e09-1c2671a84124 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.535446 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6cd99a-f603-47e0-9e3a-c99ce585a5be admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.540508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a5705-6587-4411-bfda-57abdd625c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.541211 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a5705-6587-4411-bfda-57abdd625c90 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.541497 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a5705-6587-4411-bfda-57abdd625c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.541795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a5705-6587-4411-bfda-57abdd625c90 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.542452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a5705-6587-4411-bfda-57abdd625c90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.544788 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a5705-6587-4411-bfda-57abdd625c90 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.568940 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 784/1574] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:44:02 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:02.573098 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a5705-6587-4411-bfda-57abdd625c90 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.583635 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a5705-6587-4411-bfda-57abdd625c90 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.588634 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d63dcd-5c48-4798-aba7-a2944755c365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.589272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d63dcd-5c48-4798-aba7-a2944755c365 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.589554 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d63dcd-5c48-4798-aba7-a2944755c365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.589825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d63dcd-5c48-4798-aba7-a2944755c365 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.590476 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d63dcd-5c48-4798-aba7-a2944755c365 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:44:02.590760 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b956b-0cb5-4525-bf35-6e10359547fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.590903 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d63dcd-5c48-4798-aba7-a2944755c365 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.591661 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b956b-0cb5-4525-bf35-6e10359547fa admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.592078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b956b-0cb5-4525-bf35-6e10359547fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.592360 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d63dcd-5c48-4798-aba7-a2944755c365 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.592478 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b956b-0cb5-4525-bf35-6e10359547fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.593606 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b956b-0cb5-4525-bf35-6e10359547fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.596458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b956b-0cb5-4525-bf35-6e10359547fa admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.605530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 785/1575] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:44:02.606647 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f7fb1-4c2c-4076-884c-8d9f04d7116f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.606937 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f7fb1-4c2c-4076-884c-8d9f04d7116f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.607236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f7fb1-4c2c-4076-884c-8d9f04d7116f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.628043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c51f7fb1-4c2c-4076-884c-8d9f04d7116f None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:02.629468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c51f7fb1-4c2c-4076-884c-8d9f04d7116f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:02.637977 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 786/1576] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:02.638819 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f7fb1-4c2c-4076-884c-8d9f04d7116f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.644553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 791/1577] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:44:02 2022] POST /identity/v3/projects => generated 375 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:02.646138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b956b-0cb5-4525-bf35-6e10359547fa admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.649754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f7fb1-4c2c-4076-884c-8d9f04d7116f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.707913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c12ba-ae68-41c3-805e-387ecbaba0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.708529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c12ba-ae68-41c3-805e-387ecbaba0ff admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.708936 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c12ba-ae68-41c3-805e-387ecbaba0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.709239 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c12ba-ae68-41c3-805e-387ecbaba0ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.709916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c12ba-ae68-41c3-805e-387ecbaba0ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.710603 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b956b-0cb5-4525-bf35-6e10359547fa admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.711594 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c12ba-ae68-41c3-805e-387ecbaba0ff admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.715638 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7771de-3d4c-4eba-9292-fe39984c6293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.716256 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7771de-3d4c-4eba-9292-fe39984c6293 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.716649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7771de-3d4c-4eba-9292-fe39984c6293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.716921 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7771de-3d4c-4eba-9292-fe39984c6293 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.717595 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7771de-3d4c-4eba-9292-fe39984c6293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.719512 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7771de-3d4c-4eba-9292-fe39984c6293 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.722170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 787/1578] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:02.722979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c12ba-ae68-41c3-805e-387ecbaba0ff admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.732856 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c12ba-ae68-41c3-805e-387ecbaba0ff admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.747143 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c2c41-b87c-4121-87a0-29b0a7d195bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.747816 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c2c41-b87c-4121-87a0-29b0a7d195bd admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.748084 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c2c41-b87c-4121-87a0-29b0a7d195bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.748348 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c2c41-b87c-4121-87a0-29b0a7d195bd admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.748993 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c2c41-b87c-4121-87a0-29b0a7d195bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.751041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c2c41-b87c-4121-87a0-29b0a7d195bd admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.752624 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 792/1579] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:44:02 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:02.756325 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7771de-3d4c-4eba-9292-fe39984c6293 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.777210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 788/1580] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:44:02 2022] POST /identity/v3/users => generated 285 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:02.777210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c2c41-b87c-4121-87a0-29b0a7d195bd admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.784438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c2c41-b87c-4121-87a0-29b0a7d195bd admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.786662 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:02.819434 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7771de-3d4c-4eba-9292-fe39984c6293 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.824199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec85a1-feb2-42cd-b296-d1326abf5572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.824811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec85a1-feb2-42cd-b296-d1326abf5572 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.825089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec85a1-feb2-42cd-b296-d1326abf5572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.825380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec85a1-feb2-42cd-b296-d1326abf5572 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.826022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec85a1-feb2-42cd-b296-d1326abf5572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.828002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec85a1-feb2-42cd-b296-d1326abf5572 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.850128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554a167-e771-4a91-81cb-71370ea6e7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.850838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554a167-e771-4a91-81cb-71370ea6e7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.851057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554a167-e771-4a91-81cb-71370ea6e7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.851334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554a167-e771-4a91-81cb-71370ea6e7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.851546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 793/1581] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:44:02 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:02.852138 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554a167-e771-4a91-81cb-71370ea6e7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.852411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec85a1-feb2-42cd-b296-d1326abf5572 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.853819 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554a167-e771-4a91-81cb-71370ea6e7d0 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.863259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec85a1-feb2-42cd-b296-d1326abf5572 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.864941 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 789/1582] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:02.865940 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554a167-e771-4a91-81cb-71370ea6e7d0 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.871257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab64065-5f0b-4042-bcf5-36fce52cd39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.871834 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab64065-5f0b-4042-bcf5-36fce52cd39a admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.872121 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab64065-5f0b-4042-bcf5-36fce52cd39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.872394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab64065-5f0b-4042-bcf5-36fce52cd39a admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.873005 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab64065-5f0b-4042-bcf5-36fce52cd39a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.875005 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab64065-5f0b-4042-bcf5-36fce52cd39a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.876469 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554a167-e771-4a91-81cb-71370ea6e7d0 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.881416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fa70e-924f-4bf2-af7c-7ed04c649b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.882017 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fa70e-924f-4bf2-af7c-7ed04c649b8b admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.882297 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fa70e-924f-4bf2-af7c-7ed04c649b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.882567 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fa70e-924f-4bf2-af7c-7ed04c649b8b admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:02.883202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32fa70e-924f-4bf2-af7c-7ed04c649b8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:02.885119 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32fa70e-924f-4bf2-af7c-7ed04c649b8b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:02.905172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 794/1583] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:44:02 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:02.905172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab64065-5f0b-4042-bcf5-36fce52cd39a admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.906869 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5ab64065-5f0b-4042-bcf5-36fce52cd39a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:02.933715 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 790/1584] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:44:02 2022] POST /identity/v3/users => generated 288 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:02.933715 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fa70e-924f-4bf2-af7c-7ed04c649b8b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:02.943876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fa70e-924f-4bf2-af7c-7ed04c649b8b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.954643 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:02.975245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab64065-5f0b-4042-bcf5-36fce52cd39a admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:02.979992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf329c-6753-45e3-af16-bfd912f8374a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:02.981187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf329c-6753-45e3-af16-bfd912f8374a service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:02.981508 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf329c-6753-45e3-af16-bfd912f8374a service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:02.981785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf329c-6753-45e3-af16-bfd912f8374a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.011601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33974384-ba98-493c-8edc-92ca718bbb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.012284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974384-ba98-493c-8edc-92ca718bbb37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.012557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974384-ba98-493c-8edc-92ca718bbb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.012829 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974384-ba98-493c-8edc-92ca718bbb37 admin admin] PATH_INFO: `/v3/projects/9f1cfa2fa11a49268ed10b8b0bb21f80/users/7d1d25a5745b49b383a77d48ab5ef400/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.029045 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33974384-ba98-493c-8edc-92ca718bbb37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1cfa2fa11a49268ed10b8b0bb21f80, user_id=7d1d25a5745b49b383a77d48ab5ef400, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.031631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33974384-ba98-493c-8edc-92ca718bbb37 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.044867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf329c-6753-45e3-af16-bfd912f8374a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.046388 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf329c-6753-45e3-af16-bfd912f8374a service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.050191 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 791/1585] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:02 2022] PUT /identity/v3/projects/9f1cfa2fa11a49268ed10b8b0bb21f80/users/7d1d25a5745b49b383a77d48ab5ef400/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:03.050958 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33974384-ba98-493c-8edc-92ca718bbb37 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.067906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 795/1586] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:03.068623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf329c-6753-45e3-af16-bfd912f8374a service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.078475 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf329c-6753-45e3-af16-bfd912f8374a service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.120421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33974384-ba98-493c-8edc-92ca718bbb37 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.125709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186b3b8-836a-491b-a1ee-07246420134b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.126584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186b3b8-836a-491b-a1ee-07246420134b admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.127035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186b3b8-836a-491b-a1ee-07246420134b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.127451 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186b3b8-836a-491b-a1ee-07246420134b admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.128098 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186b3b8-836a-491b-a1ee-07246420134b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.130190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186b3b8-836a-491b-a1ee-07246420134b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.131677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:03.147987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19136db-18f0-475b-944c-8ddbaaf041f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.148921 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19136db-18f0-475b-944c-8ddbaaf041f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.149247 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19136db-18f0-475b-944c-8ddbaaf041f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.149510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19136db-18f0-475b-944c-8ddbaaf041f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.150185 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19136db-18f0-475b-944c-8ddbaaf041f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.152334 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19136db-18f0-475b-944c-8ddbaaf041f0 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.161169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 792/1587] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:44:03 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:03.161169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186b3b8-836a-491b-a1ee-07246420134b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.167533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 796/1588] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:03.168516 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19136db-18f0-475b-944c-8ddbaaf041f0 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.175511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186b3b8-836a-491b-a1ee-07246420134b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.180288 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19136db-18f0-475b-944c-8ddbaaf041f0 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.182797 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0451e-7d72-465b-a12a-aa9982f033ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.183836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0451e-7d72-465b-a12a-aa9982f033ef admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.184357 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0451e-7d72-465b-a12a-aa9982f033ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.184899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0451e-7d72-465b-a12a-aa9982f033ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.185360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7371f99-1407-4ca8-8a73-f70fd13d4f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.185992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7371f99-1407-4ca8-8a73-f70fd13d4f97 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.186278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7371f99-1407-4ca8-8a73-f70fd13d4f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.186534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0451e-7d72-465b-a12a-aa9982f033ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.186639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7371f99-1407-4ca8-8a73-f70fd13d4f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.187177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7371f99-1407-4ca8-8a73-f70fd13d4f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.188767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7371f99-1407-4ca8-8a73-f70fd13d4f97 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.189184 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0451e-7d72-465b-a12a-aa9982f033ef admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.199781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 797/1589] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:03.200615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7371f99-1407-4ca8-8a73-f70fd13d4f97 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.202413 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 793/1590] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:03.203468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0451e-7d72-465b-a12a-aa9982f033ef admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.216237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0451e-7d72-465b-a12a-aa9982f033ef admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.221116 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7371f99-1407-4ca8-8a73-f70fd13d4f97 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.221534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536de8d1-f2f2-488e-b2f3-644dcead046e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.222156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536de8d1-f2f2-488e-b2f3-644dcead046e admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.222429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536de8d1-f2f2-488e-b2f3-644dcead046e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.222697 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536de8d1-f2f2-488e-b2f3-644dcead046e admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.223573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536de8d1-f2f2-488e-b2f3-644dcead046e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.225000 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536de8d1-f2f2-488e-b2f3-644dcead046e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.232568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be7cb0-53b7-4533-8823-010229bea83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.233223 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be7cb0-53b7-4533-8823-010229bea83c admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.233606 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be7cb0-53b7-4533-8823-010229bea83c admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.233751 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be7cb0-53b7-4533-8823-010229bea83c admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.234958 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5be7cb0-53b7-4533-8823-010229bea83c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.237410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5be7cb0-53b7-4533-8823-010229bea83c admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.241993 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 794/1591] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:03.244011 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7269b9-387c-4c2a-ad78-2db6cbdf11ac None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.244594 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7269b9-387c-4c2a-ad78-2db6cbdf11ac None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.245099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7269b9-387c-4c2a-ad78-2db6cbdf11ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.268209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 798/1592] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:44:03 2022] POST /identity/v3/users => generated 295 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:03.269207 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be7cb0-53b7-4533-8823-010229bea83c admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.281994 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be7cb0-53b7-4533-8823-010229bea83c admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.288210 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd52e1c-47e2-4c1f-a9bd-8297621934a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.289173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd52e1c-47e2-4c1f-a9bd-8297621934a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.290405 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd52e1c-47e2-4c1f-a9bd-8297621934a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.290951 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd52e1c-47e2-4c1f-a9bd-8297621934a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.291685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd52e1c-47e2-4c1f-a9bd-8297621934a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.293631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd52e1c-47e2-4c1f-a9bd-8297621934a2 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.304685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 799/1593] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:03.308810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-fa7269b9-387c-4c2a-ad78-2db6cbdf11ac None None] MFA Rules not processed for user `7d1d25a5745b49b383a77d48ab5ef400`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:03.308962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd52e1c-47e2-4c1f-a9bd-8297621934a2 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.311216 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-fa7269b9-387c-4c2a-ad78-2db6cbdf11ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:03.319173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd52e1c-47e2-4c1f-a9bd-8297621934a2 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.323890 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671ddcf-11f6-4126-8b4b-70c37e24257d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.324879 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671ddcf-11f6-4126-8b4b-70c37e24257d admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.324879 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671ddcf-11f6-4126-8b4b-70c37e24257d admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.325085 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671ddcf-11f6-4126-8b4b-70c37e24257d admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.325745 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671ddcf-11f6-4126-8b4b-70c37e24257d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.327354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671ddcf-11f6-4126-8b4b-70c37e24257d admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.338337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 800/1594] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:03.339223 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671ddcf-11f6-4126-8b4b-70c37e24257d admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.351747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671ddcf-11f6-4126-8b4b-70c37e24257d admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.357009 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df154701-b2c3-443f-b26a-2a30098258f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.357934 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df154701-b2c3-443f-b26a-2a30098258f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.358366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df154701-b2c3-443f-b26a-2a30098258f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.358779 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df154701-b2c3-443f-b26a-2a30098258f9 admin admin] PATH_INFO: `/v3/projects/8c72d43256114ca298ccbbd9a6da1eb1/users/4235d35b5b8a4cf6a7055b8b81f3ad8b/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.378139 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df154701-b2c3-443f-b26a-2a30098258f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c72d43256114ca298ccbbd9a6da1eb1, user_id=4235d35b5b8a4cf6a7055b8b81f3ad8b, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.380869 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df154701-b2c3-443f-b26a-2a30098258f9 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.383088 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:03.396469 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 801/1595] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:03 2022] PUT /identity/v3/projects/8c72d43256114ca298ccbbd9a6da1eb1/users/4235d35b5b8a4cf6a7055b8b81f3ad8b/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:03.397365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df154701-b2c3-443f-b26a-2a30098258f9 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.401320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 795/1596] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:03.402855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7269b9-387c-4c2a-ad78-2db6cbdf11ac None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.415660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7269b9-387c-4c2a-ad78-2db6cbdf11ac None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.486273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df154701-b2c3-443f-b26a-2a30098258f9 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.491750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220d8e6-d6c0-4724-8e68-cdc99390f211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.492367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220d8e6-d6c0-4724-8e68-cdc99390f211 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.492639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220d8e6-d6c0-4724-8e68-cdc99390f211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.492905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220d8e6-d6c0-4724-8e68-cdc99390f211 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.493578 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220d8e6-d6c0-4724-8e68-cdc99390f211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.495194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220d8e6-d6c0-4724-8e68-cdc99390f211 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.507854 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 802/1597] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:03.509477 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220d8e6-d6c0-4724-8e68-cdc99390f211 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.522802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220d8e6-d6c0-4724-8e68-cdc99390f211 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.523356 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628c7d7-f01d-4094-a60b-fd71906fadb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.524710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628c7d7-f01d-4094-a60b-fd71906fadb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.525449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628c7d7-f01d-4094-a60b-fd71906fadb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.526704 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628c7d7-f01d-4094-a60b-fd71906fadb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.527689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628c7d7-f01d-4094-a60b-fd71906fadb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.528121 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10916b37-ad3d-4674-abd8-6fabd8d6b3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.528740 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10916b37-ad3d-4674-abd8-6fabd8d6b3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.529013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10916b37-ad3d-4674-abd8-6fabd8d6b3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.529385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10916b37-ad3d-4674-abd8-6fabd8d6b3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.530099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10916b37-ad3d-4674-abd8-6fabd8d6b3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.530761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628c7d7-f01d-4094-a60b-fd71906fadb5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.531665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10916b37-ad3d-4674-abd8-6fabd8d6b3f3 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.543386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 803/1598] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:03.544850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10916b37-ad3d-4674-abd8-6fabd8d6b3f3 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.547521 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 796/1599] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:03.548563 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628c7d7-f01d-4094-a60b-fd71906fadb5 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.555469 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10916b37-ad3d-4674-abd8-6fabd8d6b3f3 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.560726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04125da7-25b2-435f-9583-aff8fa197f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.561088 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628c7d7-f01d-4094-a60b-fd71906fadb5 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.561367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04125da7-25b2-435f-9583-aff8fa197f1f admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.561640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04125da7-25b2-435f-9583-aff8fa197f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.561993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04125da7-25b2-435f-9583-aff8fa197f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.562728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04125da7-25b2-435f-9583-aff8fa197f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.564324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04125da7-25b2-435f-9583-aff8fa197f1f admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.569096 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26690782-589e-45d5-9fc9-912896b5cf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.571674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26690782-589e-45d5-9fc9-912896b5cf43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.571674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26690782-589e-45d5-9fc9-912896b5cf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.571985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26690782-589e-45d5-9fc9-912896b5cf43 admin admin] PATH_INFO: `/v3/projects/c6743e5b460d4d8eb614163a7853bcc6/users/5d98153d965c4be7b0e3339084b2852e/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.576584 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 804/1600] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:03.578779 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa9b25-307a-401e-b307-5b3841e33030 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.578779 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa9b25-307a-401e-b307-5b3841e33030 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.578779 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa9b25-307a-401e-b307-5b3841e33030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.593630 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26690782-589e-45d5-9fc9-912896b5cf43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6743e5b460d4d8eb614163a7853bcc6, user_id=5d98153d965c4be7b0e3339084b2852e, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.596649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26690782-589e-45d5-9fc9-912896b5cf43 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.607398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-20aa9b25-307a-401e-b307-5b3841e33030 None None] MFA Rules not processed for user `4235d35b5b8a4cf6a7055b8b81f3ad8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:03.608612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-20aa9b25-307a-401e-b307-5b3841e33030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:03.623863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 797/1601] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:03 2022] PUT /identity/v3/projects/c6743e5b460d4d8eb614163a7853bcc6/users/5d98153d965c4be7b0e3339084b2852e/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:03.623863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a168de-54e3-40e8-9e98-028f36d574a9 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.623863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a168de-54e3-40e8-9e98-028f36d574a9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.623863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a168de-54e3-40e8-9e98-028f36d574a9 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.623863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 798/1602] 10.222.0.57 () {54 vars in 961 bytes} [Mon Sep 26 05:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:44:03.623863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a168de-54e3-40e8-9e98-028f36d574a9 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.656150 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:03.700853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 805/1603] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:03.701794 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa9b25-307a-401e-b307-5b3841e33030 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.704991 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a168de-54e3-40e8-9e98-028f36d574a9 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.710834 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0e5c2-3dca-4718-92b8-7c478e0a26c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.711565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0e5c2-3dca-4718-92b8-7c478e0a26c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.711787 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0e5c2-3dca-4718-92b8-7c478e0a26c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.712417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0e5c2-3dca-4718-92b8-7c478e0a26c0 admin admin] PATH_INFO: `/v3/projects/7edddf22ec7f41aea66bcf3e30a3d3ca/users/d3842855395b47a5adc10c7480b5a4ca/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.712943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa9b25-307a-401e-b307-5b3841e33030 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.720163 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cff24-e4e5-49c1-909c-eb57dba0db1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.720794 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cff24-e4e5-49c1-909c-eb57dba0db1b admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.721077 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cff24-e4e5-49c1-909c-eb57dba0db1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.721388 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cff24-e4e5-49c1-909c-eb57dba0db1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.722207 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180cff24-e4e5-49c1-909c-eb57dba0db1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.724552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180cff24-e4e5-49c1-909c-eb57dba0db1b admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.732319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0e5c2-3dca-4718-92b8-7c478e0a26c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7edddf22ec7f41aea66bcf3e30a3d3ca, user_id=d3842855395b47a5adc10c7480b5a4ca, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.735302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0e5c2-3dca-4718-92b8-7c478e0a26c0 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.740267 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 806/1604] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:03.740267 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cff24-e4e5-49c1-909c-eb57dba0db1b admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.753062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cff24-e4e5-49c1-909c-eb57dba0db1b admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.754655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 799/1605] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:03 2022] PUT /identity/v3/projects/7edddf22ec7f41aea66bcf3e30a3d3ca/users/d3842855395b47a5adc10c7480b5a4ca/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:03.759254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0e5c2-3dca-4718-92b8-7c478e0a26c0 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:03.823113 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:03.840437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335e9b6-66e8-44d8-a3f7-358ab46520c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.841499 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335e9b6-66e8-44d8-a3f7-358ab46520c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.841993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335e9b6-66e8-44d8-a3f7-358ab46520c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.842493 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335e9b6-66e8-44d8-a3f7-358ab46520c2 admin admin] PATH_INFO: `/v3/projects/9fc5a928f77549839c85f3ab7e7e13b6/users/8c26eec9a5e841f8b760cc8cef309b18/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.842930 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0e5c2-3dca-4718-92b8-7c478e0a26c0 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:03.855354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204407bf-e74f-4812-8e0f-344fc8d04fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:03.856710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204407bf-e74f-4812-8e0f-344fc8d04fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.857128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204407bf-e74f-4812-8e0f-344fc8d04fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.857431 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204407bf-e74f-4812-8e0f-344fc8d04fe2 admin admin] PATH_INFO: `/v3/projects/3fa25d71f2194e638989c1e1d9ef30da/users/bfaf6a2b8a2348b4aa603cda5fe624f5/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.863774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335e9b6-66e8-44d8-a3f7-358ab46520c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fc5a928f77549839c85f3ab7e7e13b6, user_id=8c26eec9a5e841f8b760cc8cef309b18, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.868012 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335e9b6-66e8-44d8-a3f7-358ab46520c2 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.878207 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204407bf-e74f-4812-8e0f-344fc8d04fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fa25d71f2194e638989c1e1d9ef30da, user_id=bfaf6a2b8a2348b4aa603cda5fe624f5, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:03.883034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204407bf-e74f-4812-8e0f-344fc8d04fe2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:03.886041 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 807/1606] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:03 2022] PUT /identity/v3/projects/9fc5a928f77549839c85f3ab7e7e13b6/users/8c26eec9a5e841f8b760cc8cef309b18/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:03.892183 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b058a7-5764-42a3-8914-3c38681249f0 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.892855 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b058a7-5764-42a3-8914-3c38681249f0 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.893497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b058a7-5764-42a3-8914-3c38681249f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.910534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 800/1607] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:03 2022] PUT /identity/v3/projects/3fa25d71f2194e638989c1e1d9ef30da/users/bfaf6a2b8a2348b4aa603cda5fe624f5/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:03.910534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca262131-3641-4b95-a0d4-815f6810229d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:03.910534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca262131-3641-4b95-a0d4-815f6810229d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:03.910534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca262131-3641-4b95-a0d4-815f6810229d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:03.943007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ca262131-3641-4b95-a0d4-815f6810229d None None] MFA Rules not processed for user `5d98153d965c4be7b0e3339084b2852e`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:03.944640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ca262131-3641-4b95-a0d4-815f6810229d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:03.953693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-88b058a7-5764-42a3-8914-3c38681249f0 None None] MFA Rules not processed for user `92556048cbd04c409c3f4b6fd645627f`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:03.957865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-88b058a7-5764-42a3-8914-3c38681249f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:04.049530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 801/1608] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:04.050126 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 808/1609] 10.222.0.57 () {58 vars in 1058 bytes} [Mon Sep 26 05:44:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:04.050880 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca262131-3641-4b95-a0d4-815f6810229d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:04.051413 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b058a7-5764-42a3-8914-3c38681249f0 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:04.082047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:04.126321 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b058a7-5764-42a3-8914-3c38681249f0 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:04.132908 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b89a62-4e02-44db-b8e4-1ee218021c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:04.133825 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b89a62-4e02-44db-b8e4-1ee218021c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.134197 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b89a62-4e02-44db-b8e4-1ee218021c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.134376 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b89a62-4e02-44db-b8e4-1ee218021c43 admin admin] PATH_INFO: `/v3/projects/25d426f519e8498e80c6c85158bd9042/users/9d9e9c8e26774e739695b7e4d42eeafd/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:04.155292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b89a62-4e02-44db-b8e4-1ee218021c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25d426f519e8498e80c6c85158bd9042, user_id=9d9e9c8e26774e739695b7e4d42eeafd, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:04.155757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca262131-3641-4b95-a0d4-815f6810229d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:04.158054 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b89a62-4e02-44db-b8e4-1ee218021c43 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:04.162722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc014bdf-2933-4d43-83c2-4a0d562e171e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:04.163795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc014bdf-2933-4d43-83c2-4a0d562e171e admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.164114 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc014bdf-2933-4d43-83c2-4a0d562e171e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.164522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc014bdf-2933-4d43-83c2-4a0d562e171e admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:04.165068 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc014bdf-2933-4d43-83c2-4a0d562e171e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:04.167235 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc014bdf-2933-4d43-83c2-4a0d562e171e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:04.178311 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 809/1610] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:04 2022] PUT /identity/v3/projects/25d426f519e8498e80c6c85158bd9042/users/9d9e9c8e26774e739695b7e4d42eeafd/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:04.180265 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaaea51-74b6-46dd-ae90-83e489a728b7 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.180890 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaaea51-74b6-46dd-ae90-83e489a728b7 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.181624 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaaea51-74b6-46dd-ae90-83e489a728b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:04.204384 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-acaaea51-74b6-46dd-ae90-83e489a728b7 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:04.205721 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-acaaea51-74b6-46dd-ae90-83e489a728b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:04.208479 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 802/1611] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:44:04 2022] POST /identity/v3/projects => generated 371 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:04.213062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b54a5-a880-4cbe-8440-de4bcf3ca9a3 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.213371 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b54a5-a880-4cbe-8440-de4bcf3ca9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.213652 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b54a5-a880-4cbe-8440-de4bcf3ca9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:04.246987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ae9b54a5-a880-4cbe-8440-de4bcf3ca9a3 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:04.249083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ae9b54a5-a880-4cbe-8440-de4bcf3ca9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:04.292056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 810/1612] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:04.293357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0728393-fd67-41df-8022-83bded94512f None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.293641 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0728393-fd67-41df-8022-83bded94512f None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.294050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0728393-fd67-41df-8022-83bded94512f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:04.309553 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:04.318437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-c0728393-fd67-41df-8022-83bded94512f None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:04.320171 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c0728393-fd67-41df-8022-83bded94512f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:04.333849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 811/1613] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:04.334605 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0728393-fd67-41df-8022-83bded94512f None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:04.339662 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 803/1614] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:04.340637 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b54a5-a880-4cbe-8440-de4bcf3ca9a3 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:04.364058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b54a5-a880-4cbe-8440-de4bcf3ca9a3 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:04.378627 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679903af-942b-45ad-a0f2-cc7013887374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:04.380322 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679903af-942b-45ad-a0f2-cc7013887374 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.381049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679903af-942b-45ad-a0f2-cc7013887374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.381823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679903af-942b-45ad-a0f2-cc7013887374 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:04.382849 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679903af-942b-45ad-a0f2-cc7013887374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:04.385406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679903af-942b-45ad-a0f2-cc7013887374 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:04.438888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 804/1615] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:44:04 2022] POST /identity/v3/projects => generated 369 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:04.438888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5bfcf-77a4-4482-b10b-d09972f031df None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.438888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5bfcf-77a4-4482-b10b-d09972f031df None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.438888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5bfcf-77a4-4482-b10b-d09972f031df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:04.448185 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0728393-fd67-41df-8022-83bded94512f None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:04.463305 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-50c5bfcf-77a4-4482-b10b-d09972f031df None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:04.464660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-50c5bfcf-77a4-4482-b10b-d09972f031df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:04.504316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:04.553254 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b9d92-0f1a-4352-91a0-ae62a4d2700e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:04.555021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b9d92-0f1a-4352-91a0-ae62a4d2700e service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.555301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b9d92-0f1a-4352-91a0-ae62a4d2700e service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.555573 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b9d92-0f1a-4352-91a0-ae62a4d2700e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:04.556824 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 805/1616] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:04.562672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5bfcf-77a4-4482-b10b-d09972f031df None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:04.580440 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5bfcf-77a4-4482-b10b-d09972f031df None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:04.588604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f709a33-8469-4dde-af08-02ecca869ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:04.589743 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f709a33-8469-4dde-af08-02ecca869ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.590561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f709a33-8469-4dde-af08-02ecca869ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.590689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f709a33-8469-4dde-af08-02ecca869ee7 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:04.592610 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f709a33-8469-4dde-af08-02ecca869ee7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:04.596039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f709a33-8469-4dde-af08-02ecca869ee7 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:04.630409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 806/1617] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:44:04 2022] POST /identity/v3/users => generated 292 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:04.631683 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f709a33-8469-4dde-af08-02ecca869ee7 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:04.642709 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b9d92-0f1a-4352-91a0-ae62a4d2700e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:04.645600 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b9d92-0f1a-4352-91a0-ae62a4d2700e service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:04.660670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 812/1618] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:04.661801 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b9d92-0f1a-4352-91a0-ae62a4d2700e service nova] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:04.730734 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f709a33-8469-4dde-af08-02ecca869ee7 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:04.735571 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b222f8-e2fb-4681-9c1a-1d00ae969697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:04.736254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b222f8-e2fb-4681-9c1a-1d00ae969697 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.736541 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b222f8-e2fb-4681-9c1a-1d00ae969697 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.736811 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b222f8-e2fb-4681-9c1a-1d00ae969697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:04.749770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b9d92-0f1a-4352-91a0-ae62a4d2700e service nova] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:04.750015 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b222f8-e2fb-4681-9c1a-1d00ae969697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:04.751685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b222f8-e2fb-4681-9c1a-1d00ae969697 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:04.754561 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2c41c-54b8-4e0a-b290-16ba897a0409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:04.755536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2c41c-54b8-4e0a-b290-16ba897a0409 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.755663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2c41c-54b8-4e0a-b290-16ba897a0409 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.755869 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2c41c-54b8-4e0a-b290-16ba897a0409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:04.763301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 807/1619] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:04.764140 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b222f8-e2fb-4681-9c1a-1d00ae969697 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:04.768857 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2c41c-54b8-4e0a-b290-16ba897a0409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:04.771575 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2c41c-54b8-4e0a-b290-16ba897a0409 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:04.776455 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b222f8-e2fb-4681-9c1a-1d00ae969697 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:04.788349 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37e585-517c-4d88-ae3f-22bf66f664c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:04.789051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37e585-517c-4d88-ae3f-22bf66f664c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.789329 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37e585-517c-4d88-ae3f-22bf66f664c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.789605 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37e585-517c-4d88-ae3f-22bf66f664c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:04.795592 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 813/1620] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:04.799669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2c41c-54b8-4e0a-b290-16ba897a0409 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:04.818606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37e585-517c-4d88-ae3f-22bf66f664c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:04.820090 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37e585-517c-4d88-ae3f-22bf66f664c0 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:04.833407 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2c41c-54b8-4e0a-b290-16ba897a0409 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:04.839523 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 808/1621] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:04.843809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37e585-517c-4d88-ae3f-22bf66f664c0 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:04.847218 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b5c90-551e-4e21-953f-95a9c1048c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:04.848477 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b5c90-551e-4e21-953f-95a9c1048c80 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.850133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b5c90-551e-4e21-953f-95a9c1048c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.850517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b5c90-551e-4e21-953f-95a9c1048c80 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:04.851529 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b5c90-551e-4e21-953f-95a9c1048c80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:04.856555 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37e585-517c-4d88-ae3f-22bf66f664c0 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:04.858333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b5c90-551e-4e21-953f-95a9c1048c80 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:04.862749 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d11231-498c-4534-8eb6-8d77b03a266a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:04.863636 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d11231-498c-4534-8eb6-8d77b03a266a service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.864111 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d11231-498c-4534-8eb6-8d77b03a266a service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.864599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d11231-498c-4534-8eb6-8d77b03a266a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:04.878076 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d11231-498c-4534-8eb6-8d77b03a266a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:04.881864 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d11231-498c-4534-8eb6-8d77b03a266a service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:04.898730 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 809/1622] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:04.900667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d11231-498c-4534-8eb6-8d77b03a266a service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:04.908309 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 814/1623] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:44:04 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:04.913521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b5c90-551e-4e21-953f-95a9c1048c80 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:04.915886 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d11231-498c-4534-8eb6-8d77b03a266a service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:04.924154 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486702c-cec0-4d17-b179-0737d30721c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:04.925351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486702c-cec0-4d17-b179-0737d30721c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.925851 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486702c-cec0-4d17-b179-0737d30721c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.926432 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486702c-cec0-4d17-b179-0737d30721c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:04.927380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7486702c-cec0-4d17-b179-0737d30721c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:04.932710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7486702c-cec0-4d17-b179-0737d30721c5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:04.942213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b5c90-551e-4e21-953f-95a9c1048c80 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:04.950833 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 810/1624] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:04.955845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486702c-cec0-4d17-b179-0737d30721c5 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:04.956680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642eef37-7998-4a5f-858c-31c3359e1786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:04.957774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642eef37-7998-4a5f-858c-31c3359e1786 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.958166 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642eef37-7998-4a5f-858c-31c3359e1786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.958518 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642eef37-7998-4a5f-858c-31c3359e1786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:04.959268 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642eef37-7998-4a5f-858c-31c3359e1786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:04.965391 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642eef37-7998-4a5f-858c-31c3359e1786 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:04.984485 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486702c-cec0-4d17-b179-0737d30721c5 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:04.991362 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcd722-9484-4638-96a5-285ca41e21cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:04.992387 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcd722-9484-4638-96a5-285ca41e21cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.992874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcd722-9484-4638-96a5-285ca41e21cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.993364 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcd722-9484-4638-96a5-285ca41e21cc admin admin] PATH_INFO: `/v3/projects/aa293b1ed6674ae48499044c6850081c/users/54aea2fcbd5e4dbea1efaa99f2af272d/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:04.996097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 815/1625] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:04.998123 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed18e24-7de1-4dbd-8134-6519664a1b4d None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:04.998623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed18e24-7de1-4dbd-8134-6519664a1b4d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:04.999054 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed18e24-7de1-4dbd-8134-6519664a1b4d None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:05.001075 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 816/1626] 10.222.0.57 () {54 vars in 912 bytes} [Mon Sep 26 05:44:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:44:05.001961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed18e24-7de1-4dbd-8134-6519664a1b4d None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:05.016627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed18e24-7de1-4dbd-8134-6519664a1b4d None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:05.029553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad8db0-a80e-449f-994a-f2d1be0f1a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:05.030514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad8db0-a80e-449f-994a-f2d1be0f1a53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:05.030960 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:05.031317 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad8db0-a80e-449f-994a-f2d1be0f1a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:05.031846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad8db0-a80e-449f-994a-f2d1be0f1a53 admin admin] PATH_INFO: `/v3/projects/1df1a816cda24311a065237f011338fb/users/8d99d2b383d649db81a545d81eebaa93/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:05.036922 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfcd722-9484-4638-96a5-285ca41e21cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa293b1ed6674ae48499044c6850081c, user_id=54aea2fcbd5e4dbea1efaa99f2af272d, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:05.041297 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfcd722-9484-4638-96a5-285ca41e21cc admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:05.055697 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad8db0-a80e-449f-994a-f2d1be0f1a53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1df1a816cda24311a065237f011338fb, user_id=8d99d2b383d649db81a545d81eebaa93, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:05.108367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad8db0-a80e-449f-994a-f2d1be0f1a53 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:05.134884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 811/1627] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:04 2022] PUT /identity/v3/projects/aa293b1ed6674ae48499044c6850081c/users/54aea2fcbd5e4dbea1efaa99f2af272d/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:05.134884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a1d3e-95e3-4da5-8cba-6aeb25e88c6e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:05.134884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a1d3e-95e3-4da5-8cba-6aeb25e88c6e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:05.134884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a1d3e-95e3-4da5-8cba-6aeb25e88c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:05.135544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 817/1628] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:05 2022] PUT /identity/v3/projects/1df1a816cda24311a065237f011338fb/users/8d99d2b383d649db81a545d81eebaa93/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:05.135544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad8db0-a80e-449f-994a-f2d1be0f1a53 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:05.135849 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-060a1d3e-95e3-4da5-8cba-6aeb25e88c6e None None] MFA Rules not processed for user `92556048cbd04c409c3f4b6fd645627f`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:05.135849 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-060a1d3e-95e3-4da5-8cba-6aeb25e88c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:05.189941 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:05.291533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad8db0-a80e-449f-994a-f2d1be0f1a53 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:05.293115 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 812/1629] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 26 05:44:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:05.294988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a1d3e-95e3-4da5-8cba-6aeb25e88c6e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:05.312226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421784e6-740c-41e9-88e4-740221bab564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:05.315893 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421784e6-740c-41e9-88e4-740221bab564 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:05.316142 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421784e6-740c-41e9-88e4-740221bab564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:05.316571 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421784e6-740c-41e9-88e4-740221bab564 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:05.317345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421784e6-740c-41e9-88e4-740221bab564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:05.318990 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421784e6-740c-41e9-88e4-740221bab564 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:05.325952 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a1d3e-95e3-4da5-8cba-6aeb25e88c6e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:05.337311 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdee84d-33ca-4083-89a8-0d070664a7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:05.338543 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdee84d-33ca-4083-89a8-0d070664a7de admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:05.339006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdee84d-33ca-4083-89a8-0d070664a7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:05.339455 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdee84d-33ca-4083-89a8-0d070664a7de admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:05.340710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdee84d-33ca-4083-89a8-0d070664a7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:05.343229 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdee84d-33ca-4083-89a8-0d070664a7de admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:05.354254 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 818/1630] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:05.355645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421784e6-740c-41e9-88e4-740221bab564 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:05.360205 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 813/1631] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:05.361660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdee84d-33ca-4083-89a8-0d070664a7de admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:05.386805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdee84d-33ca-4083-89a8-0d070664a7de admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:05.402992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445fc76-6c9a-4e0d-84e5-10ecc2ba8151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:05.404997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445fc76-6c9a-4e0d-84e5-10ecc2ba8151 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:05.405414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445fc76-6c9a-4e0d-84e5-10ecc2ba8151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:05.407095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445fc76-6c9a-4e0d-84e5-10ecc2ba8151 admin admin] PATH_INFO: `/v3/projects/aa293b1ed6674ae48499044c6850081c/users/54aea2fcbd5e4dbea1efaa99f2af272d/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:05.411843 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445fc76-6c9a-4e0d-84e5-10ecc2ba8151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa293b1ed6674ae48499044c6850081c, user_id=54aea2fcbd5e4dbea1efaa99f2af272d, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:05.416310 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445fc76-6c9a-4e0d-84e5-10ecc2ba8151 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:05.431934 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 814/1632] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:05 2022] PUT /identity/v3/projects/aa293b1ed6674ae48499044c6850081c/users/54aea2fcbd5e4dbea1efaa99f2af272d/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:05.433115 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445fc76-6c9a-4e0d-84e5-10ecc2ba8151 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:05.493447 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421784e6-740c-41e9-88e4-740221bab564 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:05.569454 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445fc76-6c9a-4e0d-84e5-10ecc2ba8151 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:05.578243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246d5f6-d767-48ee-a410-6bd81664a26f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:05.580144 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246d5f6-d767-48ee-a410-6bd81664a26f service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:05.580676 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246d5f6-d767-48ee-a410-6bd81664a26f service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:05.581050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246d5f6-d767-48ee-a410-6bd81664a26f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:05.583759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2204c-c886-4188-a0bf-fbfdcc1209ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:05.584517 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2204c-c886-4188-a0bf-fbfdcc1209ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:05.584798 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2204c-c886-4188-a0bf-fbfdcc1209ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:05.585073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2204c-c886-4188-a0bf-fbfdcc1209ad admin admin] PATH_INFO: `/v3/projects/1df1a816cda24311a065237f011338fb/users/8d99d2b383d649db81a545d81eebaa93/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:05.588575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2204c-c886-4188-a0bf-fbfdcc1209ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1df1a816cda24311a065237f011338fb, user_id=8d99d2b383d649db81a545d81eebaa93, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:05.596827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2204c-c886-4188-a0bf-fbfdcc1209ad admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:05.624042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 815/1633] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:05 2022] PUT /identity/v3/projects/1df1a816cda24311a065237f011338fb/users/8d99d2b383d649db81a545d81eebaa93/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:05.627384 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88070b67-24e2-47b2-83a1-991c848569b7 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:05.628690 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88070b67-24e2-47b2-83a1-991c848569b7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:05.629052 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88070b67-24e2-47b2-83a1-991c848569b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:05.692459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-88070b67-24e2-47b2-83a1-991c848569b7 None None] MFA Rules not processed for user `54aea2fcbd5e4dbea1efaa99f2af272d`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:05.696529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-88070b67-24e2-47b2-83a1-991c848569b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:05.703467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246d5f6-d767-48ee-a410-6bd81664a26f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:05.705453 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246d5f6-d767-48ee-a410-6bd81664a26f service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:05.722574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 819/1634] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:05.724366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cceecdd-8da5-4d95-8ffb-c6a742c8a228 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:05.724754 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cceecdd-8da5-4d95-8ffb-c6a742c8a228 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:05.725073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cceecdd-8da5-4d95-8ffb-c6a742c8a228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:05.783172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-2cceecdd-8da5-4d95-8ffb-c6a742c8a228 None None] MFA Rules not processed for user `8d99d2b383d649db81a545d81eebaa93`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:05.788441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2cceecdd-8da5-4d95-8ffb-c6a742c8a228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:05.807480 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:05.886669 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:05.928281 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 820/1635] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:05.943135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cceecdd-8da5-4d95-8ffb-c6a742c8a228 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:05.966329 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 816/1636] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:06.035405 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cceecdd-8da5-4d95-8ffb-c6a742c8a228 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:06.043681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc86ce3-e92f-4349-b31b-75d30ccd4100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:06.044711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc86ce3-e92f-4349-b31b-75d30ccd4100 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:06.045072 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc86ce3-e92f-4349-b31b-75d30ccd4100 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:06.045429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc86ce3-e92f-4349-b31b-75d30ccd4100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:06.060781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc86ce3-e92f-4349-b31b-75d30ccd4100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:06.062823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc86ce3-e92f-4349-b31b-75d30ccd4100 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:06.079759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 821/1637] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:44:06.154153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88070b67-24e2-47b2-83a1-991c848569b7 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:06.183524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88070b67-24e2-47b2-83a1-991c848569b7 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:06.193642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b345a-77e9-4a29-8a31-3baf18a5219f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:06.195560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b345a-77e9-4a29-8a31-3baf18a5219f service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:06.196134 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b345a-77e9-4a29-8a31-3baf18a5219f service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:06.196589 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b345a-77e9-4a29-8a31-3baf18a5219f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:06.235482 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b345a-77e9-4a29-8a31-3baf18a5219f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:06.237069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b345a-77e9-4a29-8a31-3baf18a5219f service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:06.271549 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 817/1638] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:06.588746 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc86ce3-e92f-4349-b31b-75d30ccd4100 service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:06.601351 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc86ce3-e92f-4349-b31b-75d30ccd4100 service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:06.613462 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1816e-91f7-49c3-95ba-82fd37290240 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:06.614210 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1816e-91f7-49c3-95ba-82fd37290240 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:06.615620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1816e-91f7-49c3-95ba-82fd37290240 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:06.616741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1816e-91f7-49c3-95ba-82fd37290240 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:06.836460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1816e-91f7-49c3-95ba-82fd37290240 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:06.839907 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1816e-91f7-49c3-95ba-82fd37290240 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:06.859817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 822/1639] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:07.596039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16991146-1b12-4687-9e37-12f4bc436ea7 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:07.596602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16991146-1b12-4687-9e37-12f4bc436ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:07.596797 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16991146-1b12-4687-9e37-12f4bc436ea7 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:07.599471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 818/1640] 10.222.0.57 () {54 vars in 961 bytes} [Mon Sep 26 05:44:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:44:07.606923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55e570-0d5f-4422-b625-cb1d70ffb103 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:07.607512 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55e570-0d5f-4422-b625-cb1d70ffb103 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:07.607512 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55e570-0d5f-4422-b625-cb1d70ffb103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:07.678615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-7c55e570-0d5f-4422-b625-cb1d70ffb103 None None] MFA Rules not processed for user `92556048cbd04c409c3f4b6fd645627f`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:07.682071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7c55e570-0d5f-4422-b625-cb1d70ffb103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:07.819137 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:07.922933 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 823/1641] 10.222.0.57 () {58 vars in 1058 bytes} [Mon Sep 26 05:44:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:44:07.934315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8463cd-3fb0-46ce-a9ab-2c2037cafb3b None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:07.934315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8463cd-3fb0-46ce-a9ab-2c2037cafb3b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:07.934315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8463cd-3fb0-46ce-a9ab-2c2037cafb3b None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:07.934315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 819/1642] 10.222.0.57 () {54 vars in 912 bytes} [Mon Sep 26 05:44:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:44:07.940274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee4497-78a4-4aa5-91c8-5b998ac40382 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:07.940561 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee4497-78a4-4aa5-91c8-5b998ac40382 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:07.940837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee4497-78a4-4aa5-91c8-5b998ac40382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:07.984624 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-6dee4497-78a4-4aa5-91c8-5b998ac40382 None None] MFA Rules not processed for user `92556048cbd04c409c3f4b6fd645627f`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:08.010675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6dee4497-78a4-4aa5-91c8-5b998ac40382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:08.030317 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 824/1643] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 26 05:44:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:08.053051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8463cd-3fb0-46ce-a9ab-2c2037cafb3b None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:08.071566 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8463cd-3fb0-46ce-a9ab-2c2037cafb3b None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:08.082034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcbde60-1833-4490-9dcf-aa66ebbb0e9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:08.083045 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbde60-1833-4490-9dcf-aa66ebbb0e9f service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:08.083454 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbde60-1833-4490-9dcf-aa66ebbb0e9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:08.083883 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbde60-1833-4490-9dcf-aa66ebbb0e9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:08.107987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbde60-1833-4490-9dcf-aa66ebbb0e9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:08.110336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbde60-1833-4490-9dcf-aa66ebbb0e9f service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:08.127374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 820/1644] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:10.189562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee4497-78a4-4aa5-91c8-5b998ac40382 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:10.307908 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee4497-78a4-4aa5-91c8-5b998ac40382 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:10.318558 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fcf38-eba6-4fe7-a3b8-704feb7dab83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:10.322694 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fcf38-eba6-4fe7-a3b8-704feb7dab83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:10.322977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fcf38-eba6-4fe7-a3b8-704feb7dab83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:10.323247 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fcf38-eba6-4fe7-a3b8-704feb7dab83 admin admin] PATH_INFO: `/v3/users/5d98153d965c4be7b0e3339084b2852e` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:10.325259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fcf38-eba6-4fe7-a3b8-704feb7dab83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d98153d965c4be7b0e3339084b2852e)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:10.328835 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fcf38-eba6-4fe7-a3b8-704feb7dab83 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:10.419846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6e2fcf38-eba6-4fe7-a3b8-704feb7dab83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d98153d965c4be7b0e3339084b2852e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:10.439409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6e2fcf38-eba6-4fe7-a3b8-704feb7dab83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d98153d965c4be7b0e3339084b2852e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:10.440265 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6e2fcf38-eba6-4fe7-a3b8-704feb7dab83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d98153d965c4be7b0e3339084b2852e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:10.440970 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6e2fcf38-eba6-4fe7-a3b8-704feb7dab83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d98153d965c4be7b0e3339084b2852e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:10.467127 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6e2fcf38-eba6-4fe7-a3b8-704feb7dab83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d98153d965c4be7b0e3339084b2852e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:10.499534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 825/1645] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:44:10 2022] DELETE /identity/v3/users/5d98153d965c4be7b0e3339084b2852e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:10.504177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614066fc-ceb6-4f0a-9475-f1b7e5da7213 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:10.505367 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614066fc-ceb6-4f0a-9475-f1b7e5da7213 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:10.505871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614066fc-ceb6-4f0a-9475-f1b7e5da7213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:10.568122 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-614066fc-ceb6-4f0a-9475-f1b7e5da7213 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:10.569813 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-614066fc-ceb6-4f0a-9475-f1b7e5da7213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:10.666074 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 821/1646] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:10.698225 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fcf38-eba6-4fe7-a3b8-704feb7dab83 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:10.702615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6e2fcf38-eba6-4fe7-a3b8-704feb7dab83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:10.810338 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:10.815583 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fcf38-eba6-4fe7-a3b8-704feb7dab83 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:10.825882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8fb5a8-0cde-4a08-8b00-4d01169de861 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:10.827001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fb5a8-0cde-4a08-8b00-4d01169de861 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:10.828160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fb5a8-0cde-4a08-8b00-4d01169de861 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:10.829685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fb5a8-0cde-4a08-8b00-4d01169de861 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:10.849952 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8fb5a8-0cde-4a08-8b00-4d01169de861 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:10.852169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8fb5a8-0cde-4a08-8b00-4d01169de861 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:10.872200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 826/1647] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:11.116604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614066fc-ceb6-4f0a-9475-f1b7e5da7213 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:11.118840 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-614066fc-ceb6-4f0a-9475-f1b7e5da7213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:11.153428 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614066fc-ceb6-4f0a-9475-f1b7e5da7213 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:11.171359 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e93a0-fde6-4d2d-be4f-cd5eb087625f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:11.173234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e93a0-fde6-4d2d-be4f-cd5eb087625f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:11.173712 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e93a0-fde6-4d2d-be4f-cd5eb087625f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:11.174146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e93a0-fde6-4d2d-be4f-cd5eb087625f admin admin] PATH_INFO: `/v3/users/7d1d25a5745b49b383a77d48ab5ef400` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:11.176197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e93a0-fde6-4d2d-be4f-cd5eb087625f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d1d25a5745b49b383a77d48ab5ef400)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:11.177512 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: WARNING py.warnings [None req-4b5e93a0-fde6-4d2d-be4f-cd5eb087625f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:44:11.177512 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: warnings.warn(msg) Sep 26 05:44:11.177512 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: Sep 26 05:44:11.179096 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e93a0-fde6-4d2d-be4f-cd5eb087625f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:11.375425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-4b5e93a0-fde6-4d2d-be4f-cd5eb087625f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d1d25a5745b49b383a77d48ab5ef400'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:11.399416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-4b5e93a0-fde6-4d2d-be4f-cd5eb087625f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d1d25a5745b49b383a77d48ab5ef400'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:11.400754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-4b5e93a0-fde6-4d2d-be4f-cd5eb087625f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d1d25a5745b49b383a77d48ab5ef400'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:11.401929 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-4b5e93a0-fde6-4d2d-be4f-cd5eb087625f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d1d25a5745b49b383a77d48ab5ef400'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:11.467344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-4b5e93a0-fde6-4d2d-be4f-cd5eb087625f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d1d25a5745b49b383a77d48ab5ef400'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:11.541392 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8fb5a8-0cde-4a08-8b00-4d01169de861 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:11.545738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 822/1648] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:44:11 2022] DELETE /identity/v3/users/7d1d25a5745b49b383a77d48ab5ef400 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:11.546328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4f8fb5a8-0cde-4a08-8b00-4d01169de861 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:11.549277 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebed0ff-8de6-4972-a8b0-cc73f13e92b5 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:11.549600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebed0ff-8de6-4972-a8b0-cc73f13e92b5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:11.550614 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebed0ff-8de6-4972-a8b0-cc73f13e92b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:11.604806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8fb5a8-0cde-4a08-8b00-4d01169de861 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:11.615755 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-eebed0ff-8de6-4972-a8b0-cc73f13e92b5 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:11.617757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-eebed0ff-8de6-4972-a8b0-cc73f13e92b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:11.656442 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:11.742323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 823/1649] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:11.760328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464d623-0d6d-4df2-9f92-cd8beafa6900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:11.761359 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464d623-0d6d-4df2-9f92-cd8beafa6900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:11.761644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464d623-0d6d-4df2-9f92-cd8beafa6900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:11.762305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464d623-0d6d-4df2-9f92-cd8beafa6900 admin admin] PATH_INFO: `/v3/projects/c6743e5b460d4d8eb614163a7853bcc6` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:11.763915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464d623-0d6d-4df2-9f92-cd8beafa6900 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6743e5b460d4d8eb614163a7853bcc6)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:11.765244 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: WARNING py.warnings [None req-d464d623-0d6d-4df2-9f92-cd8beafa6900 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 05:44:11.765244 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: warnings.warn(msg) Sep 26 05:44:11.765244 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: Sep 26 05:44:11.766629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464d623-0d6d-4df2-9f92-cd8beafa6900 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:11.771426 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebed0ff-8de6-4972-a8b0-cc73f13e92b5 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:11.773105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-eebed0ff-8de6-4972-a8b0-cc73f13e92b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:11.878860 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebed0ff-8de6-4972-a8b0-cc73f13e92b5 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:11.890034 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d464d623-0d6d-4df2-9f92-cd8beafa6900 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6743e5b460d4d8eb614163a7853bcc6', 'request_id': 'req-d464d623-0d6d-4df2-9f92-cd8beafa6900'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:11.931868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d464d623-0d6d-4df2-9f92-cd8beafa6900 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6743e5b460d4d8eb614163a7853bcc6', 'request_id': 'req-d464d623-0d6d-4df2-9f92-cd8beafa6900'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:11.952585 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d464d623-0d6d-4df2-9f92-cd8beafa6900 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6743e5b460d4d8eb614163a7853bcc6', 'request_id': 'req-d464d623-0d6d-4df2-9f92-cd8beafa6900'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:11.978462 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d464d623-0d6d-4df2-9f92-cd8beafa6900 admin admin] The token cache is being invalidate because project c6743e5b460d4d8eb614163a7853bcc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:44:11.979152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d464d623-0d6d-4df2-9f92-cd8beafa6900 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:11.980253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d464d623-0d6d-4df2-9f92-cd8beafa6900 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:11.984588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 827/1650] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:44:11 2022] DELETE /identity/v3/projects/c6743e5b460d4d8eb614163a7853bcc6 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:44:11.988569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464d623-0d6d-4df2-9f92-cd8beafa6900 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:11.992025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d464d623-0d6d-4df2-9f92-cd8beafa6900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:12.007990 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba271fd1-5520-4519-9b71-eb367c0e6274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:12.008638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba271fd1-5520-4519-9b71-eb367c0e6274 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:12.008920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba271fd1-5520-4519-9b71-eb367c0e6274 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:12.009224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba271fd1-5520-4519-9b71-eb367c0e6274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:12.016471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ba271fd1-5520-4519-9b71-eb367c0e6274 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:12.142280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464d623-0d6d-4df2-9f92-cd8beafa6900 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:12.152851 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f66b6-2e96-45d9-b37d-64eb5c31c54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:12.153613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f66b6-2e96-45d9-b37d-64eb5c31c54e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:12.153898 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f66b6-2e96-45d9-b37d-64eb5c31c54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:12.154170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f66b6-2e96-45d9-b37d-64eb5c31c54e admin admin] PATH_INFO: `/v3/users/8d99d2b383d649db81a545d81eebaa93` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:12.155932 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f66b6-2e96-45d9-b37d-64eb5c31c54e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d99d2b383d649db81a545d81eebaa93)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:12.158651 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba271fd1-5520-4519-9b71-eb367c0e6274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:12.159198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f66b6-2e96-45d9-b37d-64eb5c31c54e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:12.160108 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba271fd1-5520-4519-9b71-eb367c0e6274 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:12.191479 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 824/1651] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:12.205282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:12.273180 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-b63f66b6-2e96-45d9-b37d-64eb5c31c54e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d99d2b383d649db81a545d81eebaa93'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:12.289061 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-b63f66b6-2e96-45d9-b37d-64eb5c31c54e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d99d2b383d649db81a545d81eebaa93'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:12.289809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-b63f66b6-2e96-45d9-b37d-64eb5c31c54e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d99d2b383d649db81a545d81eebaa93'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:12.290569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-b63f66b6-2e96-45d9-b37d-64eb5c31c54e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d99d2b383d649db81a545d81eebaa93'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:12.303099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-b63f66b6-2e96-45d9-b37d-64eb5c31c54e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d99d2b383d649db81a545d81eebaa93'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:12.336401 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 828/1652] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:44:11 2022] DELETE /identity/v3/users/8d99d2b383d649db81a545d81eebaa93 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:14.065024 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba271fd1-5520-4519-9b71-eb367c0e6274 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:14.080818 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba271fd1-5520-4519-9b71-eb367c0e6274 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:14.143968 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:14.246202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f90bd-c3fa-4dff-8ee7-34a81fd02ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:14.246883 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f90bd-c3fa-4dff-8ee7-34a81fd02ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:14.247162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f90bd-c3fa-4dff-8ee7-34a81fd02ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:14.247498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f90bd-c3fa-4dff-8ee7-34a81fd02ad3 admin admin] PATH_INFO: `/v3/projects/1df1a816cda24311a065237f011338fb` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:14.256999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f90bd-c3fa-4dff-8ee7-34a81fd02ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1df1a816cda24311a065237f011338fb)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:14.259389 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f90bd-c3fa-4dff-8ee7-34a81fd02ad3 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:14.396850 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-da2f90bd-c3fa-4dff-8ee7-34a81fd02ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1df1a816cda24311a065237f011338fb', 'request_id': 'req-da2f90bd-c3fa-4dff-8ee7-34a81fd02ad3'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:14.412244 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-da2f90bd-c3fa-4dff-8ee7-34a81fd02ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1df1a816cda24311a065237f011338fb', 'request_id': 'req-da2f90bd-c3fa-4dff-8ee7-34a81fd02ad3'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:14.433859 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-da2f90bd-c3fa-4dff-8ee7-34a81fd02ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1df1a816cda24311a065237f011338fb', 'request_id': 'req-da2f90bd-c3fa-4dff-8ee7-34a81fd02ad3'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:14.460462 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-da2f90bd-c3fa-4dff-8ee7-34a81fd02ad3 admin admin] The token cache is being invalidate because project 1df1a816cda24311a065237f011338fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:44:14.460789 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-da2f90bd-c3fa-4dff-8ee7-34a81fd02ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:14.462794 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-da2f90bd-c3fa-4dff-8ee7-34a81fd02ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:14.473572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 825/1653] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:44:14 2022] DELETE /identity/v3/projects/1df1a816cda24311a065237f011338fb => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:14.475774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8428a-9e51-470d-99da-1964a5ca2d40 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:14.476279 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8428a-9e51-470d-99da-1964a5ca2d40 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:14.476357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8428a-9e51-470d-99da-1964a5ca2d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:14.522596 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-1eb8428a-9e51-470d-99da-1964a5ca2d40 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:14.523865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-1eb8428a-9e51-470d-99da-1964a5ca2d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:14.660321 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 829/1654] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:14.660891 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31896f5f-ee87-4fee-894c-33c564f0d797 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:14.661501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31896f5f-ee87-4fee-894c-33c564f0d797 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:14.661815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8428a-9e51-470d-99da-1964a5ca2d40 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:14.661925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31896f5f-ee87-4fee-894c-33c564f0d797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:14.663638 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-1eb8428a-9e51-470d-99da-1964a5ca2d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:14.690515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8428a-9e51-470d-99da-1964a5ca2d40 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:14.700804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:14.702718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:14.703364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:14.703851 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a admin admin] PATH_INFO: `/v3/projects/9f1cfa2fa11a49268ed10b8b0bb21f80` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:14.711741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f1cfa2fa11a49268ed10b8b0bb21f80)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:14.712959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-31896f5f-ee87-4fee-894c-33c564f0d797 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:14.717570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-31896f5f-ee87-4fee-894c-33c564f0d797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:14.718957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:14.739918 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 826/1655] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:14.744820 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31896f5f-ee87-4fee-894c-33c564f0d797 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:14.821098 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31896f5f-ee87-4fee-894c-33c564f0d797 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:14.843566 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70532d-d242-4b42-b476-2567b92fc9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:14.846941 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70532d-d242-4b42-b476-2567b92fc9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:14.847400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70532d-d242-4b42-b476-2567b92fc9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:14.848501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70532d-d242-4b42-b476-2567b92fc9d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:14.850345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70532d-d242-4b42-b476-2567b92fc9d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:44:14.850613 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70532d-d242-4b42-b476-2567b92fc9d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:14.853804 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70532d-d242-4b42-b476-2567b92fc9d3 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:14.891660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 827/1656] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:44:14.894045 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916cb7f-4fa7-4717-a92e-44ad691751bf None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:14.895330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916cb7f-4fa7-4717-a92e-44ad691751bf None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:14.895477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916cb7f-4fa7-4717-a92e-44ad691751bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:14.899963 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f1cfa2fa11a49268ed10b8b0bb21f80', 'request_id': 'req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:14.912611 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f1cfa2fa11a49268ed10b8b0bb21f80', 'request_id': 'req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:14.947137 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f1cfa2fa11a49268ed10b8b0bb21f80', 'request_id': 'req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:14.967821 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a admin admin] The token cache is being invalidate because project 9f1cfa2fa11a49268ed10b8b0bb21f80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:44:14.968127 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:14.969565 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:14.974962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 830/1657] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:44:14 2022] DELETE /identity/v3/projects/9f1cfa2fa11a49268ed10b8b0bb21f80 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:14.979032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:14.980549 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-3916cb7f-4fa7-4717-a92e-44ad691751bf None None] MFA Rules not processed for user `342ce12282264d40b2f2bb6f1fc6ba25`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:14.983183 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:14.984655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3916cb7f-4fa7-4717-a92e-44ad691751bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:15.044943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:15.124737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c32d7-d0db-4c4d-a7e6-a6c36a24b42a admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:15.136845 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218a42f-ec3d-405f-951d-b64c333bc06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:15.137628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218a42f-ec3d-405f-951d-b64c333bc06c admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:15.137971 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218a42f-ec3d-405f-951d-b64c333bc06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:15.138316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218a42f-ec3d-405f-951d-b64c333bc06c admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:15.139020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218a42f-ec3d-405f-951d-b64c333bc06c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:15.144487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218a42f-ec3d-405f-951d-b64c333bc06c admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:15.179677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 828/1658] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:15.183816 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0f2bf-5898-4322-ae04-08dc66e0aa17 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:15.184331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0f2bf-5898-4322-ae04-08dc66e0aa17 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:15.184825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0f2bf-5898-4322-ae04-08dc66e0aa17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:15.219765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 831/1659] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:44:14 2022] POST /identity/v3/projects => generated 377 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:15.223575 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd21da-b68b-4b92-b6e6-eb3d04d5e1ef None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:15.225721 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd21da-b68b-4b92-b6e6-eb3d04d5e1ef None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:15.226224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd21da-b68b-4b92-b6e6-eb3d04d5e1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:15.240643 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-9dd0f2bf-5898-4322-ae04-08dc66e0aa17 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:15.242131 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9dd0f2bf-5898-4322-ae04-08dc66e0aa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:15.281788 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-60fd21da-b68b-4b92-b6e6-eb3d04d5e1ef None None] MFA Rules not processed for user `342ce12282264d40b2f2bb6f1fc6ba25`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:15.284628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-60fd21da-b68b-4b92-b6e6-eb3d04d5e1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:15.316243 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:15.404917 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 829/1660] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:15.411512 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0f2bf-5898-4322-ae04-08dc66e0aa17 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:15.436315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0f2bf-5898-4322-ae04-08dc66e0aa17 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:15.436522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 832/1661] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:15.436522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdad28-0253-4230-af92-a43fc20e60aa None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:15.436522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdad28-0253-4230-af92-a43fc20e60aa None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:15.436522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdad28-0253-4230-af92-a43fc20e60aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:15.448374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811125b-69a9-44da-b355-740cddc27493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:15.449021 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811125b-69a9-44da-b355-740cddc27493 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:15.449923 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811125b-69a9-44da-b355-740cddc27493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:15.450810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811125b-69a9-44da-b355-740cddc27493 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:15.452796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811125b-69a9-44da-b355-740cddc27493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:15.458178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811125b-69a9-44da-b355-740cddc27493 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:15.497940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-2bcdad28-0253-4230-af92-a43fc20e60aa None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:15.502250 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2bcdad28-0253-4230-af92-a43fc20e60aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:15.520214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 830/1662] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:44:15 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:15.522416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811125b-69a9-44da-b355-740cddc27493 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:15.526064 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5811125b-69a9-44da-b355-740cddc27493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:15.530810 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 833/1663] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:15.530810 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcdad28-0253-4230-af92-a43fc20e60aa None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:15.559511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcdad28-0253-4230-af92-a43fc20e60aa None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:15.580089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308185de-77ef-4c9a-8792-e23bad33aeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:15.585433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308185de-77ef-4c9a-8792-e23bad33aeac admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:15.585719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308185de-77ef-4c9a-8792-e23bad33aeac admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:15.585989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308185de-77ef-4c9a-8792-e23bad33aeac admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:15.586685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308185de-77ef-4c9a-8792-e23bad33aeac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:15.594427 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308185de-77ef-4c9a-8792-e23bad33aeac admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:15.637534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 834/1664] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:15.638460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308185de-77ef-4c9a-8792-e23bad33aeac admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:15.659610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308185de-77ef-4c9a-8792-e23bad33aeac admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:15.707103 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a41547-dbf1-4d52-a1f1-e94ba4a1da53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:15.708251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a41547-dbf1-4d52-a1f1-e94ba4a1da53 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:15.708534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a41547-dbf1-4d52-a1f1-e94ba4a1da53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:15.708817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a41547-dbf1-4d52-a1f1-e94ba4a1da53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:15.709868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a41547-dbf1-4d52-a1f1-e94ba4a1da53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:44:15.709868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a41547-dbf1-4d52-a1f1-e94ba4a1da53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:15.711509 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a41547-dbf1-4d52-a1f1-e94ba4a1da53 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:15.758416 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 835/1665] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:44:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:44:15.761887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a41547-dbf1-4d52-a1f1-e94ba4a1da53 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:15.789315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811125b-69a9-44da-b355-740cddc27493 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:15.799654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a41547-dbf1-4d52-a1f1-e94ba4a1da53 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:15.807726 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da96002-d62e-483d-b38b-35d54f218d5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:15.808381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da96002-d62e-483d-b38b-35d54f218d5f service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:15.808660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da96002-d62e-483d-b38b-35d54f218d5f service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:15.808946 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da96002-d62e-483d-b38b-35d54f218d5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:15.826876 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19899c43-b0a0-4e1a-a295-7ebed00e020d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:15.828555 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da96002-d62e-483d-b38b-35d54f218d5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:15.829545 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19899c43-b0a0-4e1a-a295-7ebed00e020d admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:15.829865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19899c43-b0a0-4e1a-a295-7ebed00e020d admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:15.830142 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19899c43-b0a0-4e1a-a295-7ebed00e020d admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:15.830546 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da96002-d62e-483d-b38b-35d54f218d5f service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:15.832333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19899c43-b0a0-4e1a-a295-7ebed00e020d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:15.834105 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19899c43-b0a0-4e1a-a295-7ebed00e020d admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:15.866848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 836/1666] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:15.874669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19899c43-b0a0-4e1a-a295-7ebed00e020d admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:15.896957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19899c43-b0a0-4e1a-a295-7ebed00e020d admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:15.897567 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 831/1667] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:15 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:15.898327 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da96002-d62e-483d-b38b-35d54f218d5f service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:15.908957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337ed06-e9eb-48ab-866a-64b545f2e535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:15.915699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337ed06-e9eb-48ab-866a-64b545f2e535 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:15.916014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337ed06-e9eb-48ab-866a-64b545f2e535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:15.925506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337ed06-e9eb-48ab-866a-64b545f2e535 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:15.926249 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337ed06-e9eb-48ab-866a-64b545f2e535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:15.930535 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da96002-d62e-483d-b38b-35d54f218d5f service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:15.941718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337ed06-e9eb-48ab-866a-64b545f2e535 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:15.959110 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117f910-f6f9-42b7-b2e7-6a9cbb178c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:15.960218 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117f910-f6f9-42b7-b2e7-6a9cbb178c5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:15.960709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117f910-f6f9-42b7-b2e7-6a9cbb178c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:15.961205 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117f910-f6f9-42b7-b2e7-6a9cbb178c5d admin admin] PATH_INFO: `/v3/projects/e272eb6be2d147c0929fe7f69f20633b/users/6a445e2e58a74919840e8527fdc55122/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:16.028145 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117f910-f6f9-42b7-b2e7-6a9cbb178c5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e272eb6be2d147c0929fe7f69f20633b, user_id=6a445e2e58a74919840e8527fdc55122, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:16.031024 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117f910-f6f9-42b7-b2e7-6a9cbb178c5d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:16.071068 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 832/1668] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:15 2022] PUT /identity/v3/projects/e272eb6be2d147c0929fe7f69f20633b/users/6a445e2e58a74919840e8527fdc55122/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:16.073557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bacf7c-8bf2-4e90-9746-b917638ae616 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:16.073899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bacf7c-8bf2-4e90-9746-b917638ae616 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:16.074172 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bacf7c-8bf2-4e90-9746-b917638ae616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:16.114755 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 837/1669] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:44:15 2022] POST /identity/v3/projects => generated 365 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:16.117126 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337ed06-e9eb-48ab-866a-64b545f2e535 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:16.176838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-83bacf7c-8bf2-4e90-9746-b917638ae616 None None] MFA Rules not processed for user `6a445e2e58a74919840e8527fdc55122`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:16.178146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-83bacf7c-8bf2-4e90-9746-b917638ae616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:16.279507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:16.303532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337ed06-e9eb-48ab-866a-64b545f2e535 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:16.318389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91fd90-aa5f-4963-98e5-f1d0f179cb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:16.318764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91fd90-aa5f-4963-98e5-f1d0f179cb97 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:16.319010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91fd90-aa5f-4963-98e5-f1d0f179cb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:16.319281 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91fd90-aa5f-4963-98e5-f1d0f179cb97 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:16.319924 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91fd90-aa5f-4963-98e5-f1d0f179cb97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:16.323107 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91fd90-aa5f-4963-98e5-f1d0f179cb97 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:16.343604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 833/1670] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:44:16.347949 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bacf7c-8bf2-4e90-9746-b917638ae616 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:16.355944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:16.368614 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bacf7c-8bf2-4e90-9746-b917638ae616 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:16.377013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 838/1671] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:44:16 2022] POST /identity/v3/users => generated 283 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:16.381694 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91fd90-aa5f-4963-98e5-f1d0f179cb97 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:16.383002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac72f1-2fe3-4e6c-a937-e787ee1d92d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:16.383640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac72f1-2fe3-4e6c-a937-e787ee1d92d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:16.383913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac72f1-2fe3-4e6c-a937-e787ee1d92d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:16.384180 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac72f1-2fe3-4e6c-a937-e787ee1d92d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:16.384822 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac72f1-2fe3-4e6c-a937-e787ee1d92d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:16.386793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac72f1-2fe3-4e6c-a937-e787ee1d92d7 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:16.400177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91fd90-aa5f-4963-98e5-f1d0f179cb97 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:16.414769 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ae3fa-352b-498b-a566-c45e85c6b934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:16.415406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ae3fa-352b-498b-a566-c45e85c6b934 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:16.415685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ae3fa-352b-498b-a566-c45e85c6b934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:16.415967 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ae3fa-352b-498b-a566-c45e85c6b934 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:16.416607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ae3fa-352b-498b-a566-c45e85c6b934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:16.420697 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ae3fa-352b-498b-a566-c45e85c6b934 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:16.449855 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 839/1672] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:16.452543 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 834/1673] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:44:16 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:16.456173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ae3fa-352b-498b-a566-c45e85c6b934 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:16.457238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac72f1-2fe3-4e6c-a937-e787ee1d92d7 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:16.482980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ae3fa-352b-498b-a566-c45e85c6b934 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:16.596943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac72f1-2fe3-4e6c-a937-e787ee1d92d7 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:16.602527 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a82767-1831-41a1-ba35-eab06111e490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:16.603236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a82767-1831-41a1-ba35-eab06111e490 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:16.603400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a82767-1831-41a1-ba35-eab06111e490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:16.603675 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a82767-1831-41a1-ba35-eab06111e490 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:16.604456 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a82767-1831-41a1-ba35-eab06111e490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:16.606564 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a82767-1831-41a1-ba35-eab06111e490 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:16.615229 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea2143-254b-43b4-a7db-827b0bfff6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:16.616169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea2143-254b-43b4-a7db-827b0bfff6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:16.616169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea2143-254b-43b4-a7db-827b0bfff6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:16.616464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea2143-254b-43b4-a7db-827b0bfff6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:16.617014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea2143-254b-43b4-a7db-827b0bfff6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:16.618659 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea2143-254b-43b4-a7db-827b0bfff6b7 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:16.643910 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 840/1674] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:16.647128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 835/1675] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:44:16 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:16.651540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea2143-254b-43b4-a7db-827b0bfff6b7 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:16.654222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a82767-1831-41a1-ba35-eab06111e490 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:16.684894 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea2143-254b-43b4-a7db-827b0bfff6b7 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:16.687912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a82767-1831-41a1-ba35-eab06111e490 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:16.702000 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30b37b-0cba-4b28-93a8-51e597aed1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:16.706917 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30b37b-0cba-4b28-93a8-51e597aed1a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:16.706917 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30b37b-0cba-4b28-93a8-51e597aed1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:16.708517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30b37b-0cba-4b28-93a8-51e597aed1a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:16.710056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30b37b-0cba-4b28-93a8-51e597aed1a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:16.712458 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348eb0af-e343-4a8a-bc33-c93dff0ad390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:16.712458 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348eb0af-e343-4a8a-bc33-c93dff0ad390 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:16.715886 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348eb0af-e343-4a8a-bc33-c93dff0ad390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:16.716459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348eb0af-e343-4a8a-bc33-c93dff0ad390 admin admin] PATH_INFO: `/v3/projects/e8b65ece0a5b4f378cac1fbc20b00028/users/4359472061b74341b425a9191f7719ae/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:16.717796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30b37b-0cba-4b28-93a8-51e597aed1a8 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:16.753894 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 841/1676] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:16.758959 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30b37b-0cba-4b28-93a8-51e597aed1a8 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:16.767228 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348eb0af-e343-4a8a-bc33-c93dff0ad390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8b65ece0a5b4f378cac1fbc20b00028, user_id=4359472061b74341b425a9191f7719ae, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:16.771268 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348eb0af-e343-4a8a-bc33-c93dff0ad390 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:16.780243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30b37b-0cba-4b28-93a8-51e597aed1a8 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:16.796366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 836/1677] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:16 2022] PUT /identity/v3/projects/e8b65ece0a5b4f378cac1fbc20b00028/users/4359472061b74341b425a9191f7719ae/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:16.796982 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac9f45-eb76-4905-b20d-f6f03a4bc80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:16.797898 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac9f45-eb76-4905-b20d-f6f03a4bc80d admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:16.798236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac9f45-eb76-4905-b20d-f6f03a4bc80d admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:16.798572 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac9f45-eb76-4905-b20d-f6f03a4bc80d admin admin] PATH_INFO: `/v3/projects/3cb994344fd24ef0bbd051a491f0dfcc/users/e75d14ba09b4456facdba6692b80d565/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:16.808647 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f8d39-97ad-40ff-9699-cd30d8a36e1f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:16.810618 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f8d39-97ad-40ff-9699-cd30d8a36e1f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:16.811095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f8d39-97ad-40ff-9699-cd30d8a36e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:16.842425 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac9f45-eb76-4905-b20d-f6f03a4bc80d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cb994344fd24ef0bbd051a491f0dfcc, user_id=e75d14ba09b4456facdba6692b80d565, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:16.845403 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac9f45-eb76-4905-b20d-f6f03a4bc80d admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:16.872929 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 842/1678] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:16 2022] PUT /identity/v3/projects/3cb994344fd24ef0bbd051a491f0dfcc/users/e75d14ba09b4456facdba6692b80d565/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:16.874173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-da2f8d39-97ad-40ff-9699-cd30d8a36e1f None None] MFA Rules not processed for user `4359472061b74341b425a9191f7719ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:16.875478 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-da2f8d39-97ad-40ff-9699-cd30d8a36e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:16.878148 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac9f45-eb76-4905-b20d-f6f03a4bc80d admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:17.072030 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac9f45-eb76-4905-b20d-f6f03a4bc80d admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:17.085247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 837/1679] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:44:17.089776 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f8d39-97ad-40ff-9699-cd30d8a36e1f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:17.090199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2cc787-8995-4456-8db9-2ac20e20ee13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:17.091223 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cc787-8995-4456-8db9-2ac20e20ee13 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:17.092537 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cc787-8995-4456-8db9-2ac20e20ee13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:17.098271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cc787-8995-4456-8db9-2ac20e20ee13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:17.099431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2cc787-8995-4456-8db9-2ac20e20ee13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:17.109754 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2cc787-8995-4456-8db9-2ac20e20ee13 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:17.132911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f8d39-97ad-40ff-9699-cd30d8a36e1f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:17.146864 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7881d-be90-4598-8126-84223d9023f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:17.147306 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 843/1680] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:44:17.148347 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2cc787-8995-4456-8db9-2ac20e20ee13 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:17.149349 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7881d-be90-4598-8126-84223d9023f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:17.149949 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7881d-be90-4598-8126-84223d9023f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:17.150770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7881d-be90-4598-8126-84223d9023f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:17.152303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7881d-be90-4598-8126-84223d9023f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:17.156829 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7881d-be90-4598-8126-84223d9023f0 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:17.178539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2cc787-8995-4456-8db9-2ac20e20ee13 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:17.198080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedc75d-056b-4ab0-81e4-f60c7bbbe1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:17.199091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedc75d-056b-4ab0-81e4-f60c7bbbe1fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:17.199608 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedc75d-056b-4ab0-81e4-f60c7bbbe1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:17.199956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedc75d-056b-4ab0-81e4-f60c7bbbe1fc admin admin] PATH_INFO: `/v3/projects/3cb994344fd24ef0bbd051a491f0dfcc/users/e75d14ba09b4456facdba6692b80d565/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:17.204165 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedc75d-056b-4ab0-81e4-f60c7bbbe1fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cb994344fd24ef0bbd051a491f0dfcc, user_id=e75d14ba09b4456facdba6692b80d565, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:17.221700 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedc75d-056b-4ab0-81e4-f60c7bbbe1fc admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:17.260762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 838/1681] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:44:17 2022] POST /identity/v3/projects => generated 365 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:17.264641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7881d-be90-4598-8126-84223d9023f0 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:17.267001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 844/1682] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:17 2022] PUT /identity/v3/projects/3cb994344fd24ef0bbd051a491f0dfcc/users/e75d14ba09b4456facdba6692b80d565/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:17.279635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd3bd7-0ada-487d-b66f-8a37f828d435 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:17.280268 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd3bd7-0ada-487d-b66f-8a37f828d435 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:17.280639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd3bd7-0ada-487d-b66f-8a37f828d435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:17.299286 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:17.338970 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:17.349289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-f3fd3bd7-0ada-487d-b66f-8a37f828d435 None None] MFA Rules not processed for user `e75d14ba09b4456facdba6692b80d565`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:17.353383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f3fd3bd7-0ada-487d-b66f-8a37f828d435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:17.425705 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7881d-be90-4598-8126-84223d9023f0 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:17.437444 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1214b-4af2-4375-aaf5-8f24b64ab1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:17.438001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1214b-4af2-4375-aaf5-8f24b64ab1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:17.438280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1214b-4af2-4375-aaf5-8f24b64ab1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:17.438549 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1214b-4af2-4375-aaf5-8f24b64ab1a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:17.439210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1214b-4af2-4375-aaf5-8f24b64ab1a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:17.443318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1214b-4af2-4375-aaf5-8f24b64ab1a9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:17.486241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 839/1683] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:44:17 2022] POST /identity/v3/users => generated 289 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:17.487675 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02e226-c4df-4393-8a3a-62d957b93f41 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:17.487963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02e226-c4df-4393-8a3a-62d957b93f41 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:17.488282 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02e226-c4df-4393-8a3a-62d957b93f41 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:17.494905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 840/1684] 10.222.0.57 () {54 vars in 912 bytes} [Mon Sep 26 05:44:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:44:17.495786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02e226-c4df-4393-8a3a-62d957b93f41 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:17.527664 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02e226-c4df-4393-8a3a-62d957b93f41 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:17.540928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3a0e0-fbc4-4890-80d2-f11df6aea625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:17.541597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3a0e0-fbc4-4890-80d2-f11df6aea625 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:17.541875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3a0e0-fbc4-4890-80d2-f11df6aea625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:17.542163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe3a0e0-fbc4-4890-80d2-f11df6aea625 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:17.542805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe3a0e0-fbc4-4890-80d2-f11df6aea625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:17.544530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe3a0e0-fbc4-4890-80d2-f11df6aea625 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:17.557647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 845/1685] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:17.557647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7968ec0-94dc-4635-bdd2-b4595875b887 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:17.566323 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7968ec0-94dc-4635-bdd2-b4595875b887 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:17.566323 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7968ec0-94dc-4635-bdd2-b4595875b887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:17.570945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 841/1686] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:17.575696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3a0e0-fbc4-4890-80d2-f11df6aea625 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:17.580377 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5fe3a0e0-fbc4-4890-80d2-f11df6aea625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:17.662707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-a7968ec0-94dc-4635-bdd2-b4595875b887 None None] MFA Rules not processed for user `6bf0f594ce7646b7b6ddb8a8553509d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:17.663992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-a7968ec0-94dc-4635-bdd2-b4595875b887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:17.670526 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe3a0e0-fbc4-4890-80d2-f11df6aea625 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:17.682998 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c36a0-9a59-4f05-a810-bea460f9902e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:17.683666 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c36a0-9a59-4f05-a810-bea460f9902e service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:17.683947 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c36a0-9a59-4f05-a810-bea460f9902e service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:17.684222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c36a0-9a59-4f05-a810-bea460f9902e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:17.796995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 846/1687] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 26 05:44:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:17.799479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7968ec0-94dc-4635-bdd2-b4595875b887 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:17.825460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7968ec0-94dc-4635-bdd2-b4595875b887 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:17.826974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c36a0-9a59-4f05-a810-bea460f9902e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:17.828570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c36a0-9a59-4f05-a810-bea460f9902e service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:17.848278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdbfcc-266c-4813-94b6-a0a8fbc7e99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:17.850241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdbfcc-266c-4813-94b6-a0a8fbc7e99b admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:17.851797 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdbfcc-266c-4813-94b6-a0a8fbc7e99b admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:17.852218 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fdbfcc-266c-4813-94b6-a0a8fbc7e99b admin admin] PATH_INFO: `/v3/projects/2b73f22674ca42b9854570dcc555188e/users/735e6c9187614c5f8688fb5d5f3740a7/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:17.854949 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 842/1688] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:17.868609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c36a0-9a59-4f05-a810-bea460f9902e service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:17.871449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2d1c36a0-9a59-4f05-a810-bea460f9902e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:17.898056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fdbfcc-266c-4813-94b6-a0a8fbc7e99b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b73f22674ca42b9854570dcc555188e, user_id=735e6c9187614c5f8688fb5d5f3740a7, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:17.901632 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fdbfcc-266c-4813-94b6-a0a8fbc7e99b admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:17.904648 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c36a0-9a59-4f05-a810-bea460f9902e service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:17.915763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4161059-49cf-4d11-ab65-a4c65441b38e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:17.916909 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4161059-49cf-4d11-ab65-a4c65441b38e service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:17.920539 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4161059-49cf-4d11-ab65-a4c65441b38e service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:17.923083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4161059-49cf-4d11-ab65-a4c65441b38e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:17.931459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 847/1689] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:17 2022] PUT /identity/v3/projects/2b73f22674ca42b9854570dcc555188e/users/735e6c9187614c5f8688fb5d5f3740a7/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:17.935538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdbfcc-266c-4813-94b6-a0a8fbc7e99b admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:17.949293 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4161059-49cf-4d11-ab65-a4c65441b38e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:17.953056 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4161059-49cf-4d11-ab65-a4c65441b38e service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:17.964493 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 843/1690] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:18.107931 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fdbfcc-266c-4813-94b6-a0a8fbc7e99b admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:18.123200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9498c-a6a5-45e6-8e64-eedf6ca675c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:18.123988 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9498c-a6a5-45e6-8e64-eedf6ca675c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:18.124173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9498c-a6a5-45e6-8e64-eedf6ca675c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:18.124381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9498c-a6a5-45e6-8e64-eedf6ca675c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:18.125050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9498c-a6a5-45e6-8e64-eedf6ca675c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:18.127063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9498c-a6a5-45e6-8e64-eedf6ca675c4 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:18.154624 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 848/1691] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:44:18.159527 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4161059-49cf-4d11-ab65-a4c65441b38e service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:18.181585 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4161059-49cf-4d11-ab65-a4c65441b38e service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:18.207491 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c3be9-f1a6-4eae-a8cd-bd4c3d166587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:18.207491 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c3be9-f1a6-4eae-a8cd-bd4c3d166587 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:18.207491 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c3be9-f1a6-4eae-a8cd-bd4c3d166587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:18.208858 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c3be9-f1a6-4eae-a8cd-bd4c3d166587 admin admin] PATH_INFO: `/v3/projects/2b73f22674ca42b9854570dcc555188e/users/735e6c9187614c5f8688fb5d5f3740a7/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:18.215904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c3be9-f1a6-4eae-a8cd-bd4c3d166587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b73f22674ca42b9854570dcc555188e, user_id=735e6c9187614c5f8688fb5d5f3740a7, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:18.227085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c3be9-f1a6-4eae-a8cd-bd4c3d166587 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:18.287957 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 844/1692] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:18 2022] PUT /identity/v3/projects/2b73f22674ca42b9854570dcc555188e/users/735e6c9187614c5f8688fb5d5f3740a7/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:18.289059 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b043c-c16c-4290-addb-0a3dc9465ece None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:18.289370 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b043c-c16c-4290-addb-0a3dc9465ece None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:18.289647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b043c-c16c-4290-addb-0a3dc9465ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:18.320796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:18.358596 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-e08b043c-c16c-4290-addb-0a3dc9465ece None None] MFA Rules not processed for user `735e6c9187614c5f8688fb5d5f3740a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:18.370303 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-e08b043c-c16c-4290-addb-0a3dc9465ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:18.668648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 849/1693] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:44:18.690887 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c3be9-f1a6-4eae-a8cd-bd4c3d166587 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:18.732109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:18.840890 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c3be9-f1a6-4eae-a8cd-bd4c3d166587 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:18.856436 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba726f24-b84e-49ca-b12a-ddbf0594ac2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:18.857762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba726f24-b84e-49ca-b12a-ddbf0594ac2c service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:18.858810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba726f24-b84e-49ca-b12a-ddbf0594ac2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:18.859656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba726f24-b84e-49ca-b12a-ddbf0594ac2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:18.918080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba726f24-b84e-49ca-b12a-ddbf0594ac2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:18.923132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba726f24-b84e-49ca-b12a-ddbf0594ac2c service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:18.976908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 845/1694] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:19.201458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b043c-c16c-4290-addb-0a3dc9465ece None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:19.217979 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b043c-c16c-4290-addb-0a3dc9465ece None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:19.224958 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204800e3-8613-49c9-868f-368b889fb94f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:19.233243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204800e3-8613-49c9-868f-368b889fb94f service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:19.233734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204800e3-8613-49c9-868f-368b889fb94f service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:19.234224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204800e3-8613-49c9-868f-368b889fb94f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:19.420695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204800e3-8613-49c9-868f-368b889fb94f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:19.422192 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204800e3-8613-49c9-868f-368b889fb94f service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:19.459401 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 850/1695] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:19.505435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba726f24-b84e-49ca-b12a-ddbf0594ac2c service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:19.508593 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ba726f24-b84e-49ca-b12a-ddbf0594ac2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:19.747097 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba726f24-b84e-49ca-b12a-ddbf0594ac2c service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:19.762697 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc5743-7f14-4704-b15a-d329f08066bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:19.763259 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc5743-7f14-4704-b15a-d329f08066bf service placement] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:19.765552 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc5743-7f14-4704-b15a-d329f08066bf service placement] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:19.765846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc5743-7f14-4704-b15a-d329f08066bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:19.773334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7afc5743-7f14-4704-b15a-d329f08066bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:19.806101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc5743-7f14-4704-b15a-d329f08066bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:19.809527 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc5743-7f14-4704-b15a-d329f08066bf service placement] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:19.864519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 846/1696] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:20.036972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf2fd4-6bab-4b48-9fd7-6af5f910fa8a None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:20.038279 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf2fd4-6bab-4b48-9fd7-6af5f910fa8a None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:20.038723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf2fd4-6bab-4b48-9fd7-6af5f910fa8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:20.105860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-2eaf2fd4-6bab-4b48-9fd7-6af5f910fa8a None None] MFA Rules not processed for user `342ce12282264d40b2f2bb6f1fc6ba25`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:20.121551 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2eaf2fd4-6bab-4b48-9fd7-6af5f910fa8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:20.352698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 851/1697] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:22.698641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c97b5-8b47-439f-8cb2-dbe28bec783b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:22.701347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c97b5-8b47-439f-8cb2-dbe28bec783b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:22.702025 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c97b5-8b47-439f-8cb2-dbe28bec783b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:22.800205 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-524c97b5-8b47-439f-8cb2-dbe28bec783b None None] MFA Rules not processed for user `8c26eec9a5e841f8b760cc8cef309b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:22.802459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-524c97b5-8b47-439f-8cb2-dbe28bec783b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:22.939176 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 847/1698] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:22.967309 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf2fd4-6bab-4b48-9fd7-6af5f910fa8a None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:23.124676 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:23.223677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf2fd4-6bab-4b48-9fd7-6af5f910fa8a None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:23.245170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eaf346-311b-45b8-b2a0-6f1d59324b5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:23.247567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eaf346-311b-45b8-b2a0-6f1d59324b5b service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:23.248662 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eaf346-311b-45b8-b2a0-6f1d59324b5b service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:23.249587 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eaf346-311b-45b8-b2a0-6f1d59324b5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:23.281569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eaf346-311b-45b8-b2a0-6f1d59324b5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:23.287111 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eaf346-311b-45b8-b2a0-6f1d59324b5b service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:23.325852 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 852/1699] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:25.611231 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9da15-f168-4c15-9b1e-956340787384 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:25.612064 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9da15-f168-4c15-9b1e-956340787384 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:25.612561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9da15-f168-4c15-9b1e-956340787384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:25.766001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840d9b1-19b5-473d-a3c7-77c014bf1c6c None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:25.768434 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840d9b1-19b5-473d-a3c7-77c014bf1c6c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:25.769386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840d9b1-19b5-473d-a3c7-77c014bf1c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:25.785193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-88e9da15-f168-4c15-9b1e-956340787384 None None] MFA Rules not processed for user `9d9e9c8e26774e739695b7e4d42eeafd`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:25.793098 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-88e9da15-f168-4c15-9b1e-956340787384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:25.919297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d840d9b1-19b5-473d-a3c7-77c014bf1c6c None None] MFA Rules not processed for user `bfaf6a2b8a2348b4aa603cda5fe624f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:25.922469 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d840d9b1-19b5-473d-a3c7-77c014bf1c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:25.947006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:26.070938 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 848/1700] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:44:26.094022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9da15-f168-4c15-9b1e-956340787384 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:26.108379 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-88e9da15-f168-4c15-9b1e-956340787384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:26.109780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 853/1701] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:44:26.167202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840d9b1-19b5-473d-a3c7-77c014bf1c6c None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:26.174485 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9da15-f168-4c15-9b1e-956340787384 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:26.209427 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840d9b1-19b5-473d-a3c7-77c014bf1c6c None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:26.219389 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e6545-8618-46c3-b05c-df90b60cecb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:26.220335 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e6545-8618-46c3-b05c-df90b60cecb0 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:26.220803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e6545-8618-46c3-b05c-df90b60cecb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:26.221347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e6545-8618-46c3-b05c-df90b60cecb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:26.254799 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac908232-524b-41f2-b498-e81e2975c306 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:26.256760 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e6545-8618-46c3-b05c-df90b60cecb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:26.258802 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e6545-8618-46c3-b05c-df90b60cecb0 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:26.264471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac908232-524b-41f2-b498-e81e2975c306 service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:26.265044 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac908232-524b-41f2-b498-e81e2975c306 service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:26.270591 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac908232-524b-41f2-b498-e81e2975c306 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:26.305391 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 849/1702] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:44:26.321737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac908232-524b-41f2-b498-e81e2975c306 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:26.327201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac908232-524b-41f2-b498-e81e2975c306 service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:26.388720 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 854/1703] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:44:26.427519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e6545-8618-46c3-b05c-df90b60cecb0 service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:26.429541 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-014e6545-8618-46c3-b05c-df90b60cecb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:26.461695 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e6545-8618-46c3-b05c-df90b60cecb0 service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:26.475282 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34697ecb-08b8-4a42-8bd3-e421847bcfb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:26.478030 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34697ecb-08b8-4a42-8bd3-e421847bcfb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:26.478412 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34697ecb-08b8-4a42-8bd3-e421847bcfb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:26.481259 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34697ecb-08b8-4a42-8bd3-e421847bcfb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:26.711681 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34697ecb-08b8-4a42-8bd3-e421847bcfb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:26.714726 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34697ecb-08b8-4a42-8bd3-e421847bcfb0 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:26.757702 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 850/1704] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:44:28.501698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd6138-78c4-46ff-bf8d-db5d299c4488 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:28.504041 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd6138-78c4-46ff-bf8d-db5d299c4488 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:28.504589 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd6138-78c4-46ff-bf8d-db5d299c4488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:28.692224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-49bd6138-78c4-46ff-bf8d-db5d299c4488 None None] MFA Rules not processed for user `0216bf67389e4627bf953b7dcc97a88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:28.693650 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-49bd6138-78c4-46ff-bf8d-db5d299c4488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:29.000281 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 855/1705] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 26 05:44:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 491 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:29.223668 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34697ecb-08b8-4a42-8bd3-e421847bcfb0 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:29.388474 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd6138-78c4-46ff-bf8d-db5d299c4488 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:29.390344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-49bd6138-78c4-46ff-bf8d-db5d299c4488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:29.437043 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:29.542702 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34697ecb-08b8-4a42-8bd3-e421847bcfb0 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:29.570174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306508c0-dffa-4790-ac0b-2bcb9af1c5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:29.577558 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306508c0-dffa-4790-ac0b-2bcb9af1c5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:29.582098 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306508c0-dffa-4790-ac0b-2bcb9af1c5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:29.585954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306508c0-dffa-4790-ac0b-2bcb9af1c5f9 admin admin] PATH_INFO: `/v3/users/4359472061b74341b425a9191f7719ae` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:29.592805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306508c0-dffa-4790-ac0b-2bcb9af1c5f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4359472061b74341b425a9191f7719ae)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:29.605105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306508c0-dffa-4790-ac0b-2bcb9af1c5f9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:29.670133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd6138-78c4-46ff-bf8d-db5d299c4488 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:29.681071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d03eca-7b48-46e4-91ff-0aa98ff93612 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f258aba84d4dd59ebe8de973ac65c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:29.682547 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d03eca-7b48-46e4-91ff-0aa98ff93612 service swift] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:29.683217 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d03eca-7b48-46e4-91ff-0aa98ff93612 service swift] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:29.683834 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d03eca-7b48-46e4-91ff-0aa98ff93612 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:29.732280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d03eca-7b48-46e4-91ff-0aa98ff93612 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:29.737589 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d03eca-7b48-46e4-91ff-0aa98ff93612 service swift] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:29.784872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-306508c0-dffa-4790-ac0b-2bcb9af1c5f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4359472061b74341b425a9191f7719ae'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:29.786849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 856/1706] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 26 05:44:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 26 05:44:29.806481 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-306508c0-dffa-4790-ac0b-2bcb9af1c5f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4359472061b74341b425a9191f7719ae'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:29.809679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-306508c0-dffa-4790-ac0b-2bcb9af1c5f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4359472061b74341b425a9191f7719ae'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:29.815762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-306508c0-dffa-4790-ac0b-2bcb9af1c5f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4359472061b74341b425a9191f7719ae'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:29.822315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:29.867610 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-306508c0-dffa-4790-ac0b-2bcb9af1c5f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4359472061b74341b425a9191f7719ae'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:29.952740 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 851/1707] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:44:29 2022] DELETE /identity/v3/users/4359472061b74341b425a9191f7719ae => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:29.958774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a21ae4-45a6-4502-954d-aa56bc0718b9 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:29.960007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a21ae4-45a6-4502-954d-aa56bc0718b9 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:29.960680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a21ae4-45a6-4502-954d-aa56bc0718b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:30.144307 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-66a21ae4-45a6-4502-954d-aa56bc0718b9 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:30.148708 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-66a21ae4-45a6-4502-954d-aa56bc0718b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:30.432710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 857/1708] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:30.541019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306508c0-dffa-4790-ac0b-2bcb9af1c5f9 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:30.543655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-306508c0-dffa-4790-ac0b-2bcb9af1c5f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:30.803390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306508c0-dffa-4790-ac0b-2bcb9af1c5f9 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:30.825590 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9249f-6a6a-4473-a8af-d54fe1036c58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:30.826710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9249f-6a6a-4473-a8af-d54fe1036c58 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:30.827212 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9249f-6a6a-4473-a8af-d54fe1036c58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:30.827799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9249f-6a6a-4473-a8af-d54fe1036c58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:30.937244 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9249f-6a6a-4473-a8af-d54fe1036c58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:30.944149 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9249f-6a6a-4473-a8af-d54fe1036c58 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:30.970884 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0d5d4-caec-47ac-abcf-4aec5a3c04fc None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:30.972238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0d5d4-caec-47ac-abcf-4aec5a3c04fc None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:30.972935 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0d5d4-caec-47ac-abcf-4aec5a3c04fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:31.013018 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 852/1709] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:31.055179 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d4c0d5d4-caec-47ac-abcf-4aec5a3c04fc None None] MFA Rules not processed for user `8c26eec9a5e841f8b760cc8cef309b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:31.057756 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d4c0d5d4-caec-47ac-abcf-4aec5a3c04fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:31.419435 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 858/1710] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:32.228425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1af808-a76b-4732-9f40-dce8f371f361 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:32.230056 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1af808-a76b-4732-9f40-dce8f371f361 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:32.230580 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1af808-a76b-4732-9f40-dce8f371f361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:32.314907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-dc1af808-a76b-4732-9f40-dce8f371f361 None None] MFA Rules not processed for user `d3842855395b47a5adc10c7480b5a4ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:32.317111 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-dc1af808-a76b-4732-9f40-dce8f371f361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:32.528775 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 853/1711] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:32.538626 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3b1ef-3e62-4171-8e32-c33c62bd072b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:32.539549 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3b1ef-3e62-4171-8e32-c33c62bd072b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:32.540375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3b1ef-3e62-4171-8e32-c33c62bd072b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:32.578128 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-c5d3b1ef-3e62-4171-8e32-c33c62bd072b None None] MFA Rules not processed for user `d3842855395b47a5adc10c7480b5a4ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:32.579861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c5d3b1ef-3e62-4171-8e32-c33c62bd072b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:32.637177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 859/1712] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:33.104362 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a399c4-e193-4b43-a650-1ed49e9dbedf None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:33.104809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a399c4-e193-4b43-a650-1ed49e9dbedf None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:33.105229 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a399c4-e193-4b43-a650-1ed49e9dbedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:33.120681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3b1ef-3e62-4171-8e32-c33c62bd072b None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:33.123281 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c5d3b1ef-3e62-4171-8e32-c33c62bd072b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:33.168806 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-10a399c4-e193-4b43-a650-1ed49e9dbedf None None] MFA Rules not processed for user `6bf0f594ce7646b7b6ddb8a8553509d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:33.173066 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-10a399c4-e193-4b43-a650-1ed49e9dbedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:33.175191 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3b1ef-3e62-4171-8e32-c33c62bd072b None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:33.189433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:33.190850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:33.192900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:33.195027 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 854/1713] 10.222.0.25 () {58 vars in 1015 bytes} [Mon Sep 26 05:44:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:33.195930 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e admin admin] PATH_INFO: `/v3/projects/e8b65ece0a5b4f378cac1fbc20b00028` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:33.199679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196afa71-cf0b-457d-aab0-b365eb7572e2 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:33.201298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196afa71-cf0b-457d-aab0-b365eb7572e2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:33.202876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196afa71-cf0b-457d-aab0-b365eb7572e2 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:33.204233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8b65ece0a5b4f378cac1fbc20b00028)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:33.207059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 855/1714] 10.222.0.57 () {54 vars in 912 bytes} [Mon Sep 26 05:44:33 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:44:33.212677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:33.217682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de69054-ba3d-42f5-9309-fcf18015022e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:33.223284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de69054-ba3d-42f5-9309-fcf18015022e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:33.224020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de69054-ba3d-42f5-9309-fcf18015022e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:33.310472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:33.312362 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7de69054-ba3d-42f5-9309-fcf18015022e None None] MFA Rules not processed for user `6bf0f594ce7646b7b6ddb8a8553509d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:33.319394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7de69054-ba3d-42f5-9309-fcf18015022e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:33.347935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 856/1715] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 26 05:44:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:33.381947 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de69054-ba3d-42f5-9309-fcf18015022e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:33.388285 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7de69054-ba3d-42f5-9309-fcf18015022e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:33.414062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de69054-ba3d-42f5-9309-fcf18015022e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:33.432423 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2445b-4df9-4957-92d5-b7b1fe638928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:33.434631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2445b-4df9-4957-92d5-b7b1fe638928 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:33.436297 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2445b-4df9-4957-92d5-b7b1fe638928 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:33.437303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2445b-4df9-4957-92d5-b7b1fe638928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:33.490380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2445b-4df9-4957-92d5-b7b1fe638928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:33.491955 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2445b-4df9-4957-92d5-b7b1fe638928 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:33.503736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8b65ece0a5b4f378cac1fbc20b00028', 'request_id': 'req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:33.527728 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 857/1716] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:44:33.528620 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2445b-4df9-4957-92d5-b7b1fe638928 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:33.532214 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8b65ece0a5b4f378cac1fbc20b00028', 'request_id': 'req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:33.558451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8b65ece0a5b4f378cac1fbc20b00028', 'request_id': 'req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:33.589115 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2445b-4df9-4957-92d5-b7b1fe638928 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:33.598358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e admin admin] The token cache is being invalidate because project e8b65ece0a5b4f378cac1fbc20b00028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:44:33.600116 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:33.612658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:33.624525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 860/1717] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:44:33 2022] DELETE /identity/v3/projects/e8b65ece0a5b4f378cac1fbc20b00028 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:33.628012 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:33.639642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:33.689444 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:33.745481 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f82c09-9453-477e-86d9-e61a1598d7d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:33.746996 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f82c09-9453-477e-86d9-e61a1598d7d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:33.747769 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f82c09-9453-477e-86d9-e61a1598d7d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:33.748441 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f82c09-9453-477e-86d9-e61a1598d7d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:33.752247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b1f82c09-9453-477e-86d9-e61a1598d7d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:33.805994 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f82c09-9453-477e-86d9-e61a1598d7d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:33.817123 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f82c09-9453-477e-86d9-e61a1598d7d7 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:33.857812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 858/1718] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:33.862774 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f82c09-9453-477e-86d9-e61a1598d7d7 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:33.867125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b1f82c09-9453-477e-86d9-e61a1598d7d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:34.097956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e7258-1f61-48ce-bf1e-bcfb47a8049e admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:34.123824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5f85a-41ff-471d-890e-9f1c6698d00c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:34.125771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5f85a-41ff-471d-890e-9f1c6698d00c service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:34.134545 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5f85a-41ff-471d-890e-9f1c6698d00c service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:34.136171 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5f85a-41ff-471d-890e-9f1c6698d00c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:34.140251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-e7a5f85a-41ff-471d-890e-9f1c6698d00c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:34.191285 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f82c09-9453-477e-86d9-e61a1598d7d7 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:34.214622 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05bafc-d724-4fea-9afe-d71a78f52bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:34.215808 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05bafc-d724-4fea-9afe-d71a78f52bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:34.216341 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05bafc-d724-4fea-9afe-d71a78f52bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:34.216842 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05bafc-d724-4fea-9afe-d71a78f52bc1 admin admin] PATH_INFO: `/v3/users/735e6c9187614c5f8688fb5d5f3740a7` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:34.218731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05bafc-d724-4fea-9afe-d71a78f52bc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=735e6c9187614c5f8688fb5d5f3740a7)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:34.221060 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05bafc-d724-4fea-9afe-d71a78f52bc1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:34.385867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5f85a-41ff-471d-890e-9f1c6698d00c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:34.388164 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5f85a-41ff-471d-890e-9f1c6698d00c service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:34.414330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0e05bafc-d724-4fea-9afe-d71a78f52bc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '735e6c9187614c5f8688fb5d5f3740a7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:34.459689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0e05bafc-d724-4fea-9afe-d71a78f52bc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '735e6c9187614c5f8688fb5d5f3740a7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:34.462801 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0e05bafc-d724-4fea-9afe-d71a78f52bc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '735e6c9187614c5f8688fb5d5f3740a7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:34.468194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0e05bafc-d724-4fea-9afe-d71a78f52bc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '735e6c9187614c5f8688fb5d5f3740a7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:34.474990 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 861/1719] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 849 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:34.516045 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0e05bafc-d724-4fea-9afe-d71a78f52bc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '735e6c9187614c5f8688fb5d5f3740a7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:34.599108 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 859/1720] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:44:33 2022] DELETE /identity/v3/users/735e6c9187614c5f8688fb5d5f3740a7 => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:39.906972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5f85a-41ff-471d-890e-9f1c6698d00c service nova] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:39.954858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5f85a-41ff-471d-890e-9f1c6698d00c service nova] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:40.078828 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:40.274881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df0274-0713-4fd8-a9ac-55f46f3235d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:40.276227 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df0274-0713-4fd8-a9ac-55f46f3235d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:40.289905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df0274-0713-4fd8-a9ac-55f46f3235d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:40.290604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df0274-0713-4fd8-a9ac-55f46f3235d6 admin admin] PATH_INFO: `/v3/projects/2b73f22674ca42b9854570dcc555188e` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:40.293079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9df0274-0713-4fd8-a9ac-55f46f3235d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b73f22674ca42b9854570dcc555188e)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:40.297255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9df0274-0713-4fd8-a9ac-55f46f3235d6 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:40.585453 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-e9df0274-0713-4fd8-a9ac-55f46f3235d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b73f22674ca42b9854570dcc555188e', 'request_id': 'req-e9df0274-0713-4fd8-a9ac-55f46f3235d6'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:40.611239 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-e9df0274-0713-4fd8-a9ac-55f46f3235d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b73f22674ca42b9854570dcc555188e', 'request_id': 'req-e9df0274-0713-4fd8-a9ac-55f46f3235d6'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:40.636464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-e9df0274-0713-4fd8-a9ac-55f46f3235d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b73f22674ca42b9854570dcc555188e', 'request_id': 'req-e9df0274-0713-4fd8-a9ac-55f46f3235d6'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:40.666819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-e9df0274-0713-4fd8-a9ac-55f46f3235d6 admin admin] The token cache is being invalidate because project 2b73f22674ca42b9854570dcc555188e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:44:40.667513 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-e9df0274-0713-4fd8-a9ac-55f46f3235d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:40.668779 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-e9df0274-0713-4fd8-a9ac-55f46f3235d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:44:40.671571 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 862/1721] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:44:39 2022] DELETE /identity/v3/projects/2b73f22674ca42b9854570dcc555188e => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:44:40.678488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c2d23-e380-47e4-84a2-f52594379c11 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:40.679123 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c2d23-e380-47e4-84a2-f52594379c11 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:40.679123 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c2d23-e380-47e4-84a2-f52594379c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:40.723148 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ce9c2d23-e380-47e4-84a2-f52594379c11 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:40.724352 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ce9c2d23-e380-47e4-84a2-f52594379c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:40.842444 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:40.870120 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 860/1722] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:40.881546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d51975-a400-4118-bbf6-dd9a9138c0f1 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:40.882011 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d51975-a400-4118-bbf6-dd9a9138c0f1 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:40.883530 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d51975-a400-4118-bbf6-dd9a9138c0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:40.948525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-39d51975-a400-4118-bbf6-dd9a9138c0f1 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:40.950627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-39d51975-a400-4118-bbf6-dd9a9138c0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:40.978796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 863/1723] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:40.986109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c2d23-e380-47e4-84a2-f52594379c11 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:41.002149 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c2d23-e380-47e4-84a2-f52594379c11 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:41.016511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9553b-345f-4fb1-94cc-9c08cbe63338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:41.017264 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9553b-345f-4fb1-94cc-9c08cbe63338 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:41.017574 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9553b-345f-4fb1-94cc-9c08cbe63338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:41.017846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9553b-345f-4fb1-94cc-9c08cbe63338 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:41.018567 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9553b-345f-4fb1-94cc-9c08cbe63338 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:44:41.018830 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9553b-345f-4fb1-94cc-9c08cbe63338 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:41.026252 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9553b-345f-4fb1-94cc-9c08cbe63338 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:41.083663 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 861/1724] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:44:41.084576 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d51975-a400-4118-bbf6-dd9a9138c0f1 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:41.103009 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d51975-a400-4118-bbf6-dd9a9138c0f1 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:41.121920 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d792b-b1f7-40a6-9ffb-46b365405823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:41.123038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d792b-b1f7-40a6-9ffb-46b365405823 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:41.125444 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d792b-b1f7-40a6-9ffb-46b365405823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:41.125915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d792b-b1f7-40a6-9ffb-46b365405823 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:41.127615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d792b-b1f7-40a6-9ffb-46b365405823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:41.131905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d792b-b1f7-40a6-9ffb-46b365405823 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:41.212016 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 864/1725] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:44:41 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:41.226307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9553b-345f-4fb1-94cc-9c08cbe63338 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:41.272766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9553b-345f-4fb1-94cc-9c08cbe63338 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:41.406357 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f2d18-0c0f-458f-bc7c-b053569bc3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:41.407576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f2d18-0c0f-458f-bc7c-b053569bc3b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:41.408143 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f2d18-0c0f-458f-bc7c-b053569bc3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:41.408654 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f2d18-0c0f-458f-bc7c-b053569bc3b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:41.409679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f2d18-0c0f-458f-bc7c-b053569bc3b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:41.412982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f2d18-0c0f-458f-bc7c-b053569bc3b1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:41.425923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d7d5d-bebb-4658-bdff-cd21470d2800 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:41.426398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d7d5d-bebb-4658-bdff-cd21470d2800 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:41.429676 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d7d5d-bebb-4658-bdff-cd21470d2800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:41.469720 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 862/1726] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:44:41 2022] POST /identity/v3/users => generated 287 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:41.471781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f2d18-0c0f-458f-bc7c-b053569bc3b1 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:41.481613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-731d7d5d-bebb-4658-bdff-cd21470d2800 None None] MFA Rules not processed for user `bfaf6a2b8a2348b4aa603cda5fe624f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:41.491311 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-731d7d5d-bebb-4658-bdff-cd21470d2800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:41.493473 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f2d18-0c0f-458f-bc7c-b053569bc3b1 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:41.501589 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cbe80-1c6e-4e05-beb9-2efcdb268002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:41.503144 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cbe80-1c6e-4e05-beb9-2efcdb268002 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:41.503624 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cbe80-1c6e-4e05-beb9-2efcdb268002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:41.504112 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cbe80-1c6e-4e05-beb9-2efcdb268002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:41.506206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cbe80-1c6e-4e05-beb9-2efcdb268002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:41.508706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cbe80-1c6e-4e05-beb9-2efcdb268002 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:41.528657 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 863/1727] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:41.533850 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cbe80-1c6e-4e05-beb9-2efcdb268002 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:41.582060 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cbe80-1c6e-4e05-beb9-2efcdb268002 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:41.604888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846001b8-358a-4af1-883d-26ff396cefc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:41.605719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846001b8-358a-4af1-883d-26ff396cefc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:41.606106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846001b8-358a-4af1-883d-26ff396cefc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:41.607200 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846001b8-358a-4af1-883d-26ff396cefc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:41.608070 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846001b8-358a-4af1-883d-26ff396cefc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:41.609945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846001b8-358a-4af1-883d-26ff396cefc5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:41.649229 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 864/1728] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:41.650411 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846001b8-358a-4af1-883d-26ff396cefc5 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:41.671997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846001b8-358a-4af1-883d-26ff396cefc5 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:41.690342 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88ea1a-edbe-4770-ac5e-d014ece2aea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:41.692270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88ea1a-edbe-4770-ac5e-d014ece2aea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:41.692892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88ea1a-edbe-4770-ac5e-d014ece2aea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:41.693436 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88ea1a-edbe-4770-ac5e-d014ece2aea1 admin admin] PATH_INFO: `/v3/projects/ee2bc9500eb9402f880f67c361e37fbb/users/b7dd624acd6f43dbba714e8bb10fa95a/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:41.743417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88ea1a-edbe-4770-ac5e-d014ece2aea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee2bc9500eb9402f880f67c361e37fbb, user_id=b7dd624acd6f43dbba714e8bb10fa95a, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:41.746405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88ea1a-edbe-4770-ac5e-d014ece2aea1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:41.793177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 865/1729] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:41.798055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 865/1730] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:41 2022] PUT /identity/v3/projects/ee2bc9500eb9402f880f67c361e37fbb/users/b7dd624acd6f43dbba714e8bb10fa95a/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:41.803879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd65e2-cff4-459e-b3de-ed7e340fd16c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:41.804291 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd65e2-cff4-459e-b3de-ed7e340fd16c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:41.804677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd65e2-cff4-459e-b3de-ed7e340fd16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:41.811402 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d09ce-f59d-494c-9466-eb69faa0ec4f None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:41.811770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d09ce-f59d-494c-9466-eb69faa0ec4f None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:41.812050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d09ce-f59d-494c-9466-eb69faa0ec4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:41.885576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-44cd65e2-cff4-459e-b3de-ed7e340fd16c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:41.887689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-44cd65e2-cff4-459e-b3de-ed7e340fd16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:41.898155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-978d09ce-f59d-494c-9466-eb69faa0ec4f None None] MFA Rules not processed for user `9d9e9c8e26774e739695b7e4d42eeafd`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:41.899450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-978d09ce-f59d-494c-9466-eb69faa0ec4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:42.057316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:42.114242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 866/1731] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:42.138912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 866/1732] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:41 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:42.150213 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd65e2-cff4-459e-b3de-ed7e340fd16c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:42.152577 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-44cd65e2-cff4-459e-b3de-ed7e340fd16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:42.178388 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fbd3e-40df-4ca1-acb5-d45eeb4c7b02 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:42.178666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fbd3e-40df-4ca1-acb5-d45eeb4c7b02 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:42.179104 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fbd3e-40df-4ca1-acb5-d45eeb4c7b02 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:42.180801 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 867/1733] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 26 05:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 05:44:42.213334 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893d296-9972-4f12-8b35-1f72ced21020 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:42.219006 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893d296-9972-4f12-8b35-1f72ced21020 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:42.219428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893d296-9972-4f12-8b35-1f72ced21020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:42.258401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:42.263385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-4893d296-9972-4f12-8b35-1f72ced21020 None None] MFA Rules not processed for user `92556048cbd04c409c3f4b6fd645627f`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:42.264735 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4893d296-9972-4f12-8b35-1f72ced21020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:42.317597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd65e2-cff4-459e-b3de-ed7e340fd16c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:42.321631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ed15f-6417-475b-9d49-dae1417f49ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:42.322279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ed15f-6417-475b-9d49-dae1417f49ea service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:42.322560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ed15f-6417-475b-9d49-dae1417f49ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:42.322832 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ed15f-6417-475b-9d49-dae1417f49ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:42.337944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ed15f-6417-475b-9d49-dae1417f49ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:42.339475 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ed15f-6417-475b-9d49-dae1417f49ea service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:42.353909 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 867/1734] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:42.433564 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 868/1735] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 26 05:44:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:44:42.456750 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2737c-618c-41ac-a166-675d3610133a None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:42.457118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2737c-618c-41ac-a166-675d3610133a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:42.457405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2737c-618c-41ac-a166-675d3610133a None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:42.459006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 868/1736] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 26 05:44:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:44:42.695814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c48c7-f434-4b81-9dc0-eb2c3a96360a None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:42.696416 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c48c7-f434-4b81-9dc0-eb2c3a96360a None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:42.696990 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c48c7-f434-4b81-9dc0-eb2c3a96360a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:42.747767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-b36c48c7-f434-4b81-9dc0-eb2c3a96360a None None] MFA Rules not processed for user `92556048cbd04c409c3f4b6fd645627f`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:42.749663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-b36c48c7-f434-4b81-9dc0-eb2c3a96360a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:42.766576 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 869/1737] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 26 05:44:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:44.346761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2737c-618c-41ac-a166-675d3610133a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:44.356432 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-31c2737c-618c-41ac-a166-675d3610133a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:44.465481 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2737c-618c-41ac-a166-675d3610133a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:44.515619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a1e0b-8240-463d-8a02-8e8cfef484d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:44.517375 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a1e0b-8240-463d-8a02-8e8cfef484d0 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:44.520921 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a1e0b-8240-463d-8a02-8e8cfef484d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:44.523264 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a1e0b-8240-463d-8a02-8e8cfef484d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:44.584329 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a1e0b-8240-463d-8a02-8e8cfef484d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:44.609434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a1e0b-8240-463d-8a02-8e8cfef484d0 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:44.717648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2184efe-5f6a-47af-82ba-a12706c69a4c None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:44.718982 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2184efe-5f6a-47af-82ba-a12706c69a4c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:44.719638 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2184efe-5f6a-47af-82ba-a12706c69a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:44.783534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 869/1738] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:44.873853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-c2184efe-5f6a-47af-82ba-a12706c69a4c None None] MFA Rules not processed for user `b7dd624acd6f43dbba714e8bb10fa95a`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:44.878328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c2184efe-5f6a-47af-82ba-a12706c69a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:45.161833 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a1e0b-8240-463d-8a02-8e8cfef484d0 service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:45.164069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6a5a1e0b-8240-463d-8a02-8e8cfef484d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:45.296346 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a1e0b-8240-463d-8a02-8e8cfef484d0 service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:45.305242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 870/1739] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 587 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:44:45.311126 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2184efe-5f6a-47af-82ba-a12706c69a4c None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:45.381426 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2184efe-5f6a-47af-82ba-a12706c69a4c None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:45.384515 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14d4e0-4d3b-4ced-962f-a3dec81f9fc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:45.392073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14d4e0-4d3b-4ced-962f-a3dec81f9fc3 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:45.393417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14d4e0-4d3b-4ced-962f-a3dec81f9fc3 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:45.397316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14d4e0-4d3b-4ced-962f-a3dec81f9fc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:45.412274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fd90e-eaa1-42f9-bbd3-905883cf0057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:45.418208 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fd90e-eaa1-42f9-bbd3-905883cf0057 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:45.419993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fd90e-eaa1-42f9-bbd3-905883cf0057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:45.422205 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fd90e-eaa1-42f9-bbd3-905883cf0057 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:45.424859 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fd90e-eaa1-42f9-bbd3-905883cf0057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:45.440447 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fd90e-eaa1-42f9-bbd3-905883cf0057 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:45.478659 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14d4e0-4d3b-4ced-962f-a3dec81f9fc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:45.490155 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14d4e0-4d3b-4ced-962f-a3dec81f9fc3 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:45.576435 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 871/1740] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:44:45 2022] POST /identity/v3/projects => generated 373 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:45.582598 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fd90e-eaa1-42f9-bbd3-905883cf0057 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:45.606692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 870/1741] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:44:45.694398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:44:45.793219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fd90e-eaa1-42f9-bbd3-905883cf0057 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:45.809047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460de43-77a0-4ea0-8709-615cb0733ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:45.811874 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460de43-77a0-4ea0-8709-615cb0733ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:45.813382 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460de43-77a0-4ea0-8709-615cb0733ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:45.814530 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460de43-77a0-4ea0-8709-615cb0733ef9 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:45.821499 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460de43-77a0-4ea0-8709-615cb0733ef9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:45.828330 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460de43-77a0-4ea0-8709-615cb0733ef9 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:45.898484 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 872/1742] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:44:45 2022] POST /identity/v3/users => generated 293 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:44:45.904888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14d4e0-4d3b-4ced-962f-a3dec81f9fc3 service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:45.967040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14d4e0-4d3b-4ced-962f-a3dec81f9fc3 service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:46.061690 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95c24a-adcc-4d27-bf3b-23f71ff077bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:46.062543 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95c24a-adcc-4d27-bf3b-23f71ff077bc admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:46.063065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95c24a-adcc-4d27-bf3b-23f71ff077bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:46.063526 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95c24a-adcc-4d27-bf3b-23f71ff077bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:46.066569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95c24a-adcc-4d27-bf3b-23f71ff077bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:46.070433 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95c24a-adcc-4d27-bf3b-23f71ff077bc admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:46.124432 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460de43-77a0-4ea0-8709-615cb0733ef9 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:46.130106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 871/1743] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:46.158521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460de43-77a0-4ea0-8709-615cb0733ef9 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:46.176942 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6fb5d-09e1-4cb6-af94-5257eecdae3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:46.178711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6fb5d-09e1-4cb6-af94-5257eecdae3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:46.181793 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6fb5d-09e1-4cb6-af94-5257eecdae3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:46.182553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6fb5d-09e1-4cb6-af94-5257eecdae3e admin admin] PATH_INFO: `/v3/projects/7260bbfc59c640458e83ed0c0cdbfd5d/users/dce6e36226d644e4a399aaff7d15ce03/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:46.295707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d6fb5d-09e1-4cb6-af94-5257eecdae3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7260bbfc59c640458e83ed0c0cdbfd5d, user_id=dce6e36226d644e4a399aaff7d15ce03, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:46.314521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d6fb5d-09e1-4cb6-af94-5257eecdae3e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:46.409317 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 873/1744] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:46 2022] PUT /identity/v3/projects/7260bbfc59c640458e83ed0c0cdbfd5d/users/dce6e36226d644e4a399aaff7d15ce03/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:44:46.411125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95c24a-adcc-4d27-bf3b-23f71ff077bc admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:46.449428 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95c24a-adcc-4d27-bf3b-23f71ff077bc admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:46.856918 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71ca82-ccfb-4903-b94f-f73411e0ada2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:46.858564 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71ca82-ccfb-4903-b94f-f73411e0ada2 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:46.858950 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71ca82-ccfb-4903-b94f-f73411e0ada2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:46.859350 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71ca82-ccfb-4903-b94f-f73411e0ada2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:46.860591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71ca82-ccfb-4903-b94f-f73411e0ada2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:46.863830 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71ca82-ccfb-4903-b94f-f73411e0ada2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:46.944356 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 872/1745] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 534 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:44:46.948829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6fb5d-09e1-4cb6-af94-5257eecdae3e admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:46.966103 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6fb5d-09e1-4cb6-af94-5257eecdae3e admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:46.991552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b73ae0-0169-4bf1-8e5c-3970c5840c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:46.992813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b73ae0-0169-4bf1-8e5c-3970c5840c8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:46.993309 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b73ae0-0169-4bf1-8e5c-3970c5840c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:46.993808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b73ae0-0169-4bf1-8e5c-3970c5840c8a admin admin] PATH_INFO: `/v3/projects/7260bbfc59c640458e83ed0c0cdbfd5d/users/dce6e36226d644e4a399aaff7d15ce03/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:47.020846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b73ae0-0169-4bf1-8e5c-3970c5840c8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7260bbfc59c640458e83ed0c0cdbfd5d, user_id=dce6e36226d644e4a399aaff7d15ce03, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:47.040340 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b73ae0-0169-4bf1-8e5c-3970c5840c8a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:47.106545 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 874/1746] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:44:46 2022] PUT /identity/v3/projects/7260bbfc59c640458e83ed0c0cdbfd5d/users/dce6e36226d644e4a399aaff7d15ce03/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:44:50.329241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b11305-ecbc-4393-b651-016e5eeb6adf None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:50.330143 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b11305-ecbc-4393-b651-016e5eeb6adf None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:50.330358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b11305-ecbc-4393-b651-016e5eeb6adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:50.386854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-88b11305-ecbc-4393-b651-016e5eeb6adf None None] MFA Rules not processed for user `dce6e36226d644e4a399aaff7d15ce03`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:44:50.388453 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-88b11305-ecbc-4393-b651-016e5eeb6adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:50.561192 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 873/1747] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:44:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:44:50.584582 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b73ae0-0169-4bf1-8e5c-3970c5840c8a admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:44:50.590657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-89b73ae0-0169-4bf1-8e5c-3970c5840c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:44:50.734056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b73ae0-0169-4bf1-8e5c-3970c5840c8a admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:44:50.739118 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e1499-cf54-4549-bdf7-4aa40744da79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:44:50.740531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e1499-cf54-4549-bdf7-4aa40744da79 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:44:50.741028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e1499-cf54-4549-bdf7-4aa40744da79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:44:50.741550 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e1499-cf54-4549-bdf7-4aa40744da79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:44:50.962864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e1499-cf54-4549-bdf7-4aa40744da79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:44:50.967742 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e1499-cf54-4549-bdf7-4aa40744da79 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:44:51.030525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 875/1748] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:44:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:00.553585 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b11305-ecbc-4393-b651-016e5eeb6adf None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:00.556086 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=92 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:00.560210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-88b11305-ecbc-4393-b651-016e5eeb6adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:00.599178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b11305-ecbc-4393-b651-016e5eeb6adf None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:00.603523 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae1e68-abe0-4c1b-9244-98a6cf17a337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:00.604192 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae1e68-abe0-4c1b-9244-98a6cf17a337 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:00.604465 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae1e68-abe0-4c1b-9244-98a6cf17a337 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:00.604943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae1e68-abe0-4c1b-9244-98a6cf17a337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:00.618261 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae1e68-abe0-4c1b-9244-98a6cf17a337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:00.619694 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae1e68-abe0-4c1b-9244-98a6cf17a337 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:00.635876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 874/1749] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:00.992673 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=73,keystone:INSERT=5 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:10.626526 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=6 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:16.350355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e1499-cf54-4549-bdf7-4aa40744da79 service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:16.353002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-0c1e1499-cf54-4549-bdf7-4aa40744da79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:16.661867 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b7870-282d-42dd-82c0-7bf0348af2d0 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:16.662885 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b7870-282d-42dd-82c0-7bf0348af2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:16.663366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b7870-282d-42dd-82c0-7bf0348af2d0 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:16.665685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 875/1750] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 26 05:45:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:45:16.686637 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e1499-cf54-4549-bdf7-4aa40744da79 service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:16.719236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6aa38-213e-4edb-b797-6ed2917a5c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:16.720461 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6aa38-213e-4edb-b797-6ed2917a5c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:16.726137 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6aa38-213e-4edb-b797-6ed2917a5c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:16.726640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6aa38-213e-4edb-b797-6ed2917a5c56 admin admin] PATH_INFO: `/v3/users/6a445e2e58a74919840e8527fdc55122` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:16.737063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6aa38-213e-4edb-b797-6ed2917a5c56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a445e2e58a74919840e8527fdc55122)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:16.746757 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6aa38-213e-4edb-b797-6ed2917a5c56 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:16.842592 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef8f3b-4acd-4ffc-9649-0da19adf6d20 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:16.843166 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef8f3b-4acd-4ffc-9649-0da19adf6d20 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:16.843658 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef8f3b-4acd-4ffc-9649-0da19adf6d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:16.908243 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a1ef8f3b-4acd-4ffc-9649-0da19adf6d20 None None] MFA Rules not processed for user `92556048cbd04c409c3f4b6fd645627f`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:16.915514 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a1ef8f3b-4acd-4ffc-9649-0da19adf6d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:16.952889 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8fa6aa38-213e-4edb-b797-6ed2917a5c56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a445e2e58a74919840e8527fdc55122'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:16.989720 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8fa6aa38-213e-4edb-b797-6ed2917a5c56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a445e2e58a74919840e8527fdc55122'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:17.000675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8fa6aa38-213e-4edb-b797-6ed2917a5c56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a445e2e58a74919840e8527fdc55122'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:17.004872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8fa6aa38-213e-4edb-b797-6ed2917a5c56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a445e2e58a74919840e8527fdc55122'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:17.074123 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8fa6aa38-213e-4edb-b797-6ed2917a5c56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a445e2e58a74919840e8527fdc55122'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:17.171192 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 876/1751] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:16 2022] DELETE /identity/v3/users/6a445e2e58a74919840e8527fdc55122 => generated 0 bytes in 822 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:17.176052 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 876/1752] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 26 05:45:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:45:17.176686 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a961a72-3160-4e55-a92d-c8f495b98244 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:17.177081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a961a72-3160-4e55-a92d-c8f495b98244 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:17.177482 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a961a72-3160-4e55-a92d-c8f495b98244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:17.234453 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-5a961a72-3160-4e55-a92d-c8f495b98244 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:17.250439 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5a961a72-3160-4e55-a92d-c8f495b98244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:17.556157 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 877/1753] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:45:17.615707 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef8f3b-4acd-4ffc-9649-0da19adf6d20 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:17.617641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a1ef8f3b-4acd-4ffc-9649-0da19adf6d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:17.780502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef8f3b-4acd-4ffc-9649-0da19adf6d20 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:17.829214 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a961a72-3160-4e55-a92d-c8f495b98244 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:17.830976 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5a961a72-3160-4e55-a92d-c8f495b98244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:17.871646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:17.957591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02754867-ec1c-4dc8-8ca0-61d5da323ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:17.958566 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02754867-ec1c-4dc8-8ca0-61d5da323ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:17.958920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02754867-ec1c-4dc8-8ca0-61d5da323ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:17.959274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02754867-ec1c-4dc8-8ca0-61d5da323ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:18.001660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02754867-ec1c-4dc8-8ca0-61d5da323ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:18.009321 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02754867-ec1c-4dc8-8ca0-61d5da323ed8 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:18.040716 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 877/1754] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:18.050758 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a961a72-3160-4e55-a92d-c8f495b98244 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:18.058932 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d939533-58c4-4b4a-8930-1790d578521f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:18.059926 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d939533-58c4-4b4a-8930-1790d578521f service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:18.060369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d939533-58c4-4b4a-8930-1790d578521f service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:18.060846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d939533-58c4-4b4a-8930-1790d578521f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:18.073987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d939533-58c4-4b4a-8930-1790d578521f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:18.076278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d939533-58c4-4b4a-8930-1790d578521f service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:18.091338 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 878/1755] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:45:19.947629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02754867-ec1c-4dc8-8ca0-61d5da323ed8 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:19.951283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-02754867-ec1c-4dc8-8ca0-61d5da323ed8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:19.979283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02754867-ec1c-4dc8-8ca0-61d5da323ed8 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:19.988778 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e4015-3d23-4da2-af58-bccfec53b05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:19.991256 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e4015-3d23-4da2-af58-bccfec53b05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:19.991805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e4015-3d23-4da2-af58-bccfec53b05e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:19.992405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e4015-3d23-4da2-af58-bccfec53b05e admin admin] PATH_INFO: `/v3/projects/e272eb6be2d147c0929fe7f69f20633b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:19.995004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e4015-3d23-4da2-af58-bccfec53b05e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e272eb6be2d147c0929fe7f69f20633b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:19.999554 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e4015-3d23-4da2-af58-bccfec53b05e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:20.098558 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:20.114749 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e87e4015-3d23-4da2-af58-bccfec53b05e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e272eb6be2d147c0929fe7f69f20633b', 'request_id': 'req-e87e4015-3d23-4da2-af58-bccfec53b05e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:20.133504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e87e4015-3d23-4da2-af58-bccfec53b05e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e272eb6be2d147c0929fe7f69f20633b', 'request_id': 'req-e87e4015-3d23-4da2-af58-bccfec53b05e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:20.149045 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e87e4015-3d23-4da2-af58-bccfec53b05e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e272eb6be2d147c0929fe7f69f20633b', 'request_id': 'req-e87e4015-3d23-4da2-af58-bccfec53b05e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:20.168519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e87e4015-3d23-4da2-af58-bccfec53b05e admin admin] The token cache is being invalidate because project e272eb6be2d147c0929fe7f69f20633b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:20.169184 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e87e4015-3d23-4da2-af58-bccfec53b05e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:20.170265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e87e4015-3d23-4da2-af58-bccfec53b05e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:20.172078 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 878/1756] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:19 2022] DELETE /identity/v3/projects/e272eb6be2d147c0929fe7f69f20633b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:20.176944 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d939533-58c4-4b4a-8930-1790d578521f service nova] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:20.196015 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d939533-58c4-4b4a-8930-1790d578521f service nova] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:20.306332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f4f87-955f-4d81-8260-a22b4659fc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:20.307588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f4f87-955f-4d81-8260-a22b4659fc29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:20.308094 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f4f87-955f-4d81-8260-a22b4659fc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:20.308582 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f4f87-955f-4d81-8260-a22b4659fc29 admin admin] PATH_INFO: `/v3/users/e75d14ba09b4456facdba6692b80d565` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:20.310835 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f4f87-955f-4d81-8260-a22b4659fc29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e75d14ba09b4456facdba6692b80d565)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:20.315115 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72f4f87-955f-4d81-8260-a22b4659fc29 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:20.451661 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d72f4f87-955f-4d81-8260-a22b4659fc29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e75d14ba09b4456facdba6692b80d565'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:20.465574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d72f4f87-955f-4d81-8260-a22b4659fc29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e75d14ba09b4456facdba6692b80d565'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:20.466722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d72f4f87-955f-4d81-8260-a22b4659fc29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e75d14ba09b4456facdba6692b80d565'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:20.467955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d72f4f87-955f-4d81-8260-a22b4659fc29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e75d14ba09b4456facdba6692b80d565'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:20.482457 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d72f4f87-955f-4d81-8260-a22b4659fc29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e75d14ba09b4456facdba6692b80d565'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:20.514110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 879/1757] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:20 2022] DELETE /identity/v3/users/e75d14ba09b4456facdba6692b80d565 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:45:21.051082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e4015-3d23-4da2-af58-bccfec53b05e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:21.054999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e87e4015-3d23-4da2-af58-bccfec53b05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:21.082915 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e4015-3d23-4da2-af58-bccfec53b05e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:21.186879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acd393-b3f3-4395-817f-70a6c210aca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:21.188130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acd393-b3f3-4395-817f-70a6c210aca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:21.188727 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acd393-b3f3-4395-817f-70a6c210aca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:21.189749 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acd393-b3f3-4395-817f-70a6c210aca8 admin admin] PATH_INFO: `/v3/projects/3cb994344fd24ef0bbd051a491f0dfcc` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:21.191647 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11acd393-b3f3-4395-817f-70a6c210aca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cb994344fd24ef0bbd051a491f0dfcc)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:21.195417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11acd393-b3f3-4395-817f-70a6c210aca8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:21.307520 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-11acd393-b3f3-4395-817f-70a6c210aca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cb994344fd24ef0bbd051a491f0dfcc', 'request_id': 'req-11acd393-b3f3-4395-817f-70a6c210aca8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:21.328818 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-11acd393-b3f3-4395-817f-70a6c210aca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cb994344fd24ef0bbd051a491f0dfcc', 'request_id': 'req-11acd393-b3f3-4395-817f-70a6c210aca8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:21.347636 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-11acd393-b3f3-4395-817f-70a6c210aca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cb994344fd24ef0bbd051a491f0dfcc', 'request_id': 'req-11acd393-b3f3-4395-817f-70a6c210aca8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:21.373164 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-11acd393-b3f3-4395-817f-70a6c210aca8 admin admin] The token cache is being invalidate because project 3cb994344fd24ef0bbd051a491f0dfcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:21.373676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-11acd393-b3f3-4395-817f-70a6c210aca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:21.374828 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-11acd393-b3f3-4395-817f-70a6c210aca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:21.377820 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 879/1758] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:21 2022] DELETE /identity/v3/projects/3cb994344fd24ef0bbd051a491f0dfcc => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:21.384629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cabd3-571a-4dda-90b8-00df725f4aef None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:21.385428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cabd3-571a-4dda-90b8-00df725f4aef None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:21.385428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cabd3-571a-4dda-90b8-00df725f4aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:21.407143 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-ea9cabd3-571a-4dda-90b8-00df725f4aef None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:21.408346 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ea9cabd3-571a-4dda-90b8-00df725f4aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:21.435679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:21.499384 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 880/1759] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:45:21.513303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b7a7b-f9cf-42fc-b57a-16d745ee8e09 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:21.513764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b7a7b-f9cf-42fc-b57a-16d745ee8e09 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:21.514134 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b7a7b-f9cf-42fc-b57a-16d745ee8e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:21.540386 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f38b7a7b-f9cf-42fc-b57a-16d745ee8e09 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:21.541665 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f38b7a7b-f9cf-42fc-b57a-16d745ee8e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:21.552349 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 880/1760] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:21.558286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cabd3-571a-4dda-90b8-00df725f4aef None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:21.569605 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cabd3-571a-4dda-90b8-00df725f4aef None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:21.574491 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10e6bd-db65-4212-9b6b-1b802a234f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:21.575121 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10e6bd-db65-4212-9b6b-1b802a234f12 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:21.575401 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10e6bd-db65-4212-9b6b-1b802a234f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:21.575667 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10e6bd-db65-4212-9b6b-1b802a234f12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:21.576332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10e6bd-db65-4212-9b6b-1b802a234f12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:45:21.576634 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10e6bd-db65-4212-9b6b-1b802a234f12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:21.578354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10e6bd-db65-4212-9b6b-1b802a234f12 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:21.594678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 881/1761] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:45:21.598651 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b7a7b-f9cf-42fc-b57a-16d745ee8e09 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:21.614455 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b7a7b-f9cf-42fc-b57a-16d745ee8e09 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:21.622348 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a4e0c-a3dd-404a-aafe-30cb693b820a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:21.623438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a4e0c-a3dd-404a-aafe-30cb693b820a admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:21.623932 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a4e0c-a3dd-404a-aafe-30cb693b820a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:21.624417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a4e0c-a3dd-404a-aafe-30cb693b820a admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:21.625654 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a4e0c-a3dd-404a-aafe-30cb693b820a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:21.629505 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a4e0c-a3dd-404a-aafe-30cb693b820a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:21.685848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 881/1762] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:21 2022] POST /identity/v3/projects => generated 387 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:21.686850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10e6bd-db65-4212-9b6b-1b802a234f12 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:21.708278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10e6bd-db65-4212-9b6b-1b802a234f12 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:21.789548 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b3ec0-0aa7-4140-8b75-4cce1b5bed2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:21.790183 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b3ec0-0aa7-4140-8b75-4cce1b5bed2a admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:21.790449 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b3ec0-0aa7-4140-8b75-4cce1b5bed2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:21.790722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b3ec0-0aa7-4140-8b75-4cce1b5bed2a admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:21.791381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b3ec0-0aa7-4140-8b75-4cce1b5bed2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:21.793360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b3ec0-0aa7-4140-8b75-4cce1b5bed2a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:21.817864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 882/1763] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:21 2022] POST /identity/v3/users => generated 294 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:45:21.822474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a4e0c-a3dd-404a-aafe-30cb693b820a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:21.834962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a4e0c-a3dd-404a-aafe-30cb693b820a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:21.841077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737abb51-0d68-411d-924e-65b560f92491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:21.841754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737abb51-0d68-411d-924e-65b560f92491 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:21.842029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737abb51-0d68-411d-924e-65b560f92491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:21.842300 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737abb51-0d68-411d-924e-65b560f92491 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:21.842955 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737abb51-0d68-411d-924e-65b560f92491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:21.844593 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737abb51-0d68-411d-924e-65b560f92491 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:21.855785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 882/1764] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:45:21.862844 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b3ec0-0aa7-4140-8b75-4cce1b5bed2a admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:21.877896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b3ec0-0aa7-4140-8b75-4cce1b5bed2a admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:21.884194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34396cd4-e6a5-442a-98d7-239f40421e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:21.885442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34396cd4-e6a5-442a-98d7-239f40421e3e admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:21.885874 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34396cd4-e6a5-442a-98d7-239f40421e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:21.886332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34396cd4-e6a5-442a-98d7-239f40421e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:21.887415 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34396cd4-e6a5-442a-98d7-239f40421e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:21.890486 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34396cd4-e6a5-442a-98d7-239f40421e3e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:21.910538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 883/1765] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:21.915214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737abb51-0d68-411d-924e-65b560f92491 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:21.928101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737abb51-0d68-411d-924e-65b560f92491 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:21.936404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c273c0-4856-40db-99a3-293aed15ea5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:21.937809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c273c0-4856-40db-99a3-293aed15ea5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:21.938310 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c273c0-4856-40db-99a3-293aed15ea5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:21.938839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c273c0-4856-40db-99a3-293aed15ea5a admin admin] PATH_INFO: `/v3/projects/7b09ea3fd4ed4049b281e2ad08dd97ec/users/c393a48d4b374a1e914c01cafff622cb/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:21.965910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c273c0-4856-40db-99a3-293aed15ea5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b09ea3fd4ed4049b281e2ad08dd97ec, user_id=c393a48d4b374a1e914c01cafff622cb, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:21.968606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c273c0-4856-40db-99a3-293aed15ea5a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:21.982472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:21.991883 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 883/1766] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:21 2022] PUT /identity/v3/projects/7b09ea3fd4ed4049b281e2ad08dd97ec/users/c393a48d4b374a1e914c01cafff622cb/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:22.012931 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda4d92-3232-494c-947c-5a1add88579f None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:22.013344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda4d92-3232-494c-947c-5a1add88579f None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:22.013761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda4d92-3232-494c-947c-5a1add88579f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:22.062337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-2bda4d92-3232-494c-947c-5a1add88579f None None] MFA Rules not processed for user `c393a48d4b374a1e914c01cafff622cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:22.063688 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2bda4d92-3232-494c-947c-5a1add88579f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:22.187980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 884/1767] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:22.192822 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c273c0-4856-40db-99a3-293aed15ea5a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:22.305757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c273c0-4856-40db-99a3-293aed15ea5a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:22.313929 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62320c5f-2892-48d2-ad08-c3075277032f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:22.315011 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62320c5f-2892-48d2-ad08-c3075277032f admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:22.315491 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62320c5f-2892-48d2-ad08-c3075277032f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:22.315959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62320c5f-2892-48d2-ad08-c3075277032f admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:22.317242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62320c5f-2892-48d2-ad08-c3075277032f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:22.320999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62320c5f-2892-48d2-ad08-c3075277032f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:22.364062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 884/1768] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:22 2022] POST /identity/v3/projects => generated 387 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:22.368567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda4d92-3232-494c-947c-5a1add88579f None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:22.386457 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda4d92-3232-494c-947c-5a1add88579f None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:22.416986 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:22.477714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4eda58-1be1-41d8-a6a7-881c14240e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:22.479371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4eda58-1be1-41d8-a6a7-881c14240e98 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:22.479371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4eda58-1be1-41d8-a6a7-881c14240e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:22.479914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4eda58-1be1-41d8-a6a7-881c14240e98 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:22.485530 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4eda58-1be1-41d8-a6a7-881c14240e98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:22.487807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4eda58-1be1-41d8-a6a7-881c14240e98 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:22.514201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 885/1769] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:22 2022] POST /identity/v3/users => generated 300 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:22.519573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62320c5f-2892-48d2-ad08-c3075277032f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:22.537999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62320c5f-2892-48d2-ad08-c3075277032f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:22.543304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df6bce-d534-4fe9-800f-5c73943e1e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:22.544057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df6bce-d534-4fe9-800f-5c73943e1e50 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:22.544337 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df6bce-d534-4fe9-800f-5c73943e1e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:22.544656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df6bce-d534-4fe9-800f-5c73943e1e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:22.545623 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70df6bce-d534-4fe9-800f-5c73943e1e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:22.548044 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70df6bce-d534-4fe9-800f-5c73943e1e50 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:22.561516 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 885/1770] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:45:22.566237 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4eda58-1be1-41d8-a6a7-881c14240e98 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:22.583367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4eda58-1be1-41d8-a6a7-881c14240e98 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:22.591661 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d7525-b577-42eb-a565-bf38b4fc7fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:22.592911 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d7525-b577-42eb-a565-bf38b4fc7fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:22.593352 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d7525-b577-42eb-a565-bf38b4fc7fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:22.593762 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d7525-b577-42eb-a565-bf38b4fc7fe7 admin admin] PATH_INFO: `/v3/projects/cbdc067853214c64864b630eca8f46db/users/b2ee8b22d98d42f1817a2a13908005e0/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:22.616657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d7525-b577-42eb-a565-bf38b4fc7fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbdc067853214c64864b630eca8f46db, user_id=b2ee8b22d98d42f1817a2a13908005e0, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:22.620466 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d7525-b577-42eb-a565-bf38b4fc7fe7 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:22.645921 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 886/1771] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:22 2022] PUT /identity/v3/projects/cbdc067853214c64864b630eca8f46db/users/b2ee8b22d98d42f1817a2a13908005e0/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:45:22.650837 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df6bce-d534-4fe9-800f-5c73943e1e50 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:22.663592 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df6bce-d534-4fe9-800f-5c73943e1e50 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:22.747733 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a241fe-f703-46c1-afff-24bc96569081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:22.748815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a241fe-f703-46c1-afff-24bc96569081 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:22.749363 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a241fe-f703-46c1-afff-24bc96569081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:22.749858 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a241fe-f703-46c1-afff-24bc96569081 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:22.751070 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a241fe-f703-46c1-afff-24bc96569081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:22.754200 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a241fe-f703-46c1-afff-24bc96569081 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:22.766814 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 886/1772] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:45:22.771361 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d7525-b577-42eb-a565-bf38b4fc7fe7 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:22.782195 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d7525-b577-42eb-a565-bf38b4fc7fe7 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:22.788604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9242c-6f10-433c-92ea-17144cb01bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:22.789449 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9242c-6f10-433c-92ea-17144cb01bb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:22.789724 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9242c-6f10-433c-92ea-17144cb01bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:22.789996 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9242c-6f10-433c-92ea-17144cb01bb7 admin admin] PATH_INFO: `/v3/projects/cbdc067853214c64864b630eca8f46db/users/b2ee8b22d98d42f1817a2a13908005e0/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:22.792260 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9242c-6f10-433c-92ea-17144cb01bb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbdc067853214c64864b630eca8f46db, user_id=b2ee8b22d98d42f1817a2a13908005e0, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:22.795081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9242c-6f10-433c-92ea-17144cb01bb7 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:22.813926 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 887/1773] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:22 2022] PUT /identity/v3/projects/cbdc067853214c64864b630eca8f46db/users/b2ee8b22d98d42f1817a2a13908005e0/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:45:22.818725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47e7a7-161d-4933-b187-d96d705b3f97 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:22.819017 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47e7a7-161d-4933-b187-d96d705b3f97 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:22.819347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47e7a7-161d-4933-b187-d96d705b3f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:22.853421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-5e47e7a7-161d-4933-b187-d96d705b3f97 None None] MFA Rules not processed for user `b2ee8b22d98d42f1817a2a13908005e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:22.854781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5e47e7a7-161d-4933-b187-d96d705b3f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:22.948168 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:22.961803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 887/1774] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:22.973987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9242c-6f10-433c-92ea-17144cb01bb7 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:22.976004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-eae9242c-6f10-433c-92ea-17144cb01bb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:23.069116 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9242c-6f10-433c-92ea-17144cb01bb7 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:23.075417 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3a67b-2deb-4232-a39b-aa575678afc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:23.076486 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3a67b-2deb-4232-a39b-aa575678afc9 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:23.076964 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3a67b-2deb-4232-a39b-aa575678afc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:23.077430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3a67b-2deb-4232-a39b-aa575678afc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:23.093940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3a67b-2deb-4232-a39b-aa575678afc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:23.096596 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3a67b-2deb-4232-a39b-aa575678afc9 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:23.108369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 888/1775] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:23.192340 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47e7a7-161d-4933-b187-d96d705b3f97 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:23.291092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47e7a7-161d-4933-b187-d96d705b3f97 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:23.298678 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9bc64-91ad-434b-bd69-7deae828ccfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:23.299546 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9bc64-91ad-434b-bd69-7deae828ccfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:23.299998 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9bc64-91ad-434b-bd69-7deae828ccfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:23.300298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9bc64-91ad-434b-bd69-7deae828ccfd admin admin] PATH_INFO: `/v3/users/c393a48d4b374a1e914c01cafff622cb` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:23.303012 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e9bc64-91ad-434b-bd69-7deae828ccfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c393a48d4b374a1e914c01cafff622cb)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:23.305865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e9bc64-91ad-434b-bd69-7deae828ccfd admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:23.378922 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-01e9bc64-91ad-434b-bd69-7deae828ccfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c393a48d4b374a1e914c01cafff622cb'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:23.388757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-01e9bc64-91ad-434b-bd69-7deae828ccfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c393a48d4b374a1e914c01cafff622cb'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:23.389869 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-01e9bc64-91ad-434b-bd69-7deae828ccfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c393a48d4b374a1e914c01cafff622cb'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:23.390625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-01e9bc64-91ad-434b-bd69-7deae828ccfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c393a48d4b374a1e914c01cafff622cb'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:23.407152 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-01e9bc64-91ad-434b-bd69-7deae828ccfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c393a48d4b374a1e914c01cafff622cb'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:23.422887 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 888/1776] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:23 2022] DELETE /identity/v3/users/c393a48d4b374a1e914c01cafff622cb => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:23.427801 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d379d48-0773-4836-abb1-08e7911beefc None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:23.428393 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d379d48-0773-4836-abb1-08e7911beefc None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:23.428393 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d379d48-0773-4836-abb1-08e7911beefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:23.450257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-4d379d48-0773-4836-abb1-08e7911beefc None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:23.451811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4d379d48-0773-4836-abb1-08e7911beefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:23.530216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 889/1777] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:45:23.537649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9bc64-91ad-434b-bd69-7deae828ccfd admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:23.540008 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-01e9bc64-91ad-434b-bd69-7deae828ccfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:23.622817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9bc64-91ad-434b-bd69-7deae828ccfd admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:23.627326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcf147-e3da-4101-ae20-d558fa24981b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:23.628001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcf147-e3da-4101-ae20-d558fa24981b service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:23.628236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcf147-e3da-4101-ae20-d558fa24981b service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:23.628511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcf147-e3da-4101-ae20-d558fa24981b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:23.639628 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fcf147-e3da-4101-ae20-d558fa24981b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:23.641301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fcf147-e3da-4101-ae20-d558fa24981b service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:23.652900 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 889/1778] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:45:24.127368 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d379d48-0773-4836-abb1-08e7911beefc None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:24.130139 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4d379d48-0773-4836-abb1-08e7911beefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:24.151448 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d379d48-0773-4836-abb1-08e7911beefc None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:24.159081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:24.160039 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:24.160413 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:24.160840 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2 admin admin] PATH_INFO: `/v3/projects/7b09ea3fd4ed4049b281e2ad08dd97ec` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:24.165252 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b09ea3fd4ed4049b281e2ad08dd97ec)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:24.165252 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:24.178218 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:24.246358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b09ea3fd4ed4049b281e2ad08dd97ec', 'request_id': 'req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:24.259109 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b09ea3fd4ed4049b281e2ad08dd97ec', 'request_id': 'req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:24.270831 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b09ea3fd4ed4049b281e2ad08dd97ec', 'request_id': 'req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:24.287987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2 admin admin] The token cache is being invalidate because project 7b09ea3fd4ed4049b281e2ad08dd97ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:24.288185 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:24.288877 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:24.290599 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 890/1779] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:24 2022] DELETE /identity/v3/projects/7b09ea3fd4ed4049b281e2ad08dd97ec => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:24.295811 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcf147-e3da-4101-ae20-d558fa24981b service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:24.314907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcf147-e3da-4101-ae20-d558fa24981b service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:24.338449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:24.411752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65cce3-cab7-4450-97ca-1add5383b47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:24.412687 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65cce3-cab7-4450-97ca-1add5383b47b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:24.413452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65cce3-cab7-4450-97ca-1add5383b47b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:24.414069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65cce3-cab7-4450-97ca-1add5383b47b admin admin] PATH_INFO: `/v3/users/b2ee8b22d98d42f1817a2a13908005e0` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:24.416126 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65cce3-cab7-4450-97ca-1add5383b47b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2ee8b22d98d42f1817a2a13908005e0)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:24.420110 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65cce3-cab7-4450-97ca-1add5383b47b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:24.494673 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2d65cce3-cab7-4450-97ca-1add5383b47b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2ee8b22d98d42f1817a2a13908005e0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:24.507030 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2d65cce3-cab7-4450-97ca-1add5383b47b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2ee8b22d98d42f1817a2a13908005e0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:24.508024 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2d65cce3-cab7-4450-97ca-1add5383b47b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2ee8b22d98d42f1817a2a13908005e0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:24.509083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2d65cce3-cab7-4450-97ca-1add5383b47b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2ee8b22d98d42f1817a2a13908005e0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:24.529091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-2d65cce3-cab7-4450-97ca-1add5383b47b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2ee8b22d98d42f1817a2a13908005e0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:24.549823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 890/1780] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:24 2022] DELETE /identity/v3/users/b2ee8b22d98d42f1817a2a13908005e0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:26.013815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:26.016046 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:26.030658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457591c8-e594-4ae1-a9d1-dcd5fac1bcc2 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:26.176314 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca6423-f88c-4a34-833e-d422f8ab4738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:26.177371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca6423-f88c-4a34-833e-d422f8ab4738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:26.177828 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca6423-f88c-4a34-833e-d422f8ab4738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:26.179214 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca6423-f88c-4a34-833e-d422f8ab4738 admin admin] PATH_INFO: `/v3/projects/cbdc067853214c64864b630eca8f46db` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:26.181526 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca6423-f88c-4a34-833e-d422f8ab4738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbdc067853214c64864b630eca8f46db)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:26.186328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca6423-f88c-4a34-833e-d422f8ab4738 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:26.290115 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7bca6423-f88c-4a34-833e-d422f8ab4738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbdc067853214c64864b630eca8f46db', 'request_id': 'req-7bca6423-f88c-4a34-833e-d422f8ab4738'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:26.306221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7bca6423-f88c-4a34-833e-d422f8ab4738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbdc067853214c64864b630eca8f46db', 'request_id': 'req-7bca6423-f88c-4a34-833e-d422f8ab4738'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:26.327549 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7bca6423-f88c-4a34-833e-d422f8ab4738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbdc067853214c64864b630eca8f46db', 'request_id': 'req-7bca6423-f88c-4a34-833e-d422f8ab4738'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:26.367969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7bca6423-f88c-4a34-833e-d422f8ab4738 admin admin] The token cache is being invalidate because project cbdc067853214c64864b630eca8f46db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:26.368522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7bca6423-f88c-4a34-833e-d422f8ab4738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:26.369811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7bca6423-f88c-4a34-833e-d422f8ab4738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:26.377806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 891/1781] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:26 2022] DELETE /identity/v3/projects/cbdc067853214c64864b630eca8f46db => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:26.380611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd336e-4e04-4d0b-b548-6592a144ab7f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:26.381427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd336e-4e04-4d0b-b548-6592a144ab7f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:26.382924 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd336e-4e04-4d0b-b548-6592a144ab7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:26.449542 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-1fdd336e-4e04-4d0b-b548-6592a144ab7f None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:26.453364 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1fdd336e-4e04-4d0b-b548-6592a144ab7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:26.697387 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 891/1782] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:26.701477 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd0e67-bad7-4fd2-9edd-15a0c085ab86 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:26.702116 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd0e67-bad7-4fd2-9edd-15a0c085ab86 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:26.702237 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd0e67-bad7-4fd2-9edd-15a0c085ab86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:26.755418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-dacd0e67-bad7-4fd2-9edd-15a0c085ab86 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:26.759145 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-dacd0e67-bad7-4fd2-9edd-15a0c085ab86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:26.786014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 892/1783] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:26.795326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd336e-4e04-4d0b-b548-6592a144ab7f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:26.821364 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd336e-4e04-4d0b-b548-6592a144ab7f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:26.832963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3fa844-0450-4848-a8cb-4162c1861c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:26.835243 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fa844-0450-4848-a8cb-4162c1861c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:26.835670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fa844-0450-4848-a8cb-4162c1861c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:26.836160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fa844-0450-4848-a8cb-4162c1861c2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:26.838717 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3fa844-0450-4848-a8cb-4162c1861c2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:45:26.839240 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3fa844-0450-4848-a8cb-4162c1861c2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:26.843377 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3fa844-0450-4848-a8cb-4162c1861c2b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:26.876189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 892/1784] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:45:26.884943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd0e67-bad7-4fd2-9edd-15a0c085ab86 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:26.904852 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd0e67-bad7-4fd2-9edd-15a0c085ab86 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:26.911069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e7380-c9ef-40b9-8667-7369b035c598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:26.911812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e7380-c9ef-40b9-8667-7369b035c598 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:26.912135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e7380-c9ef-40b9-8667-7369b035c598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:26.912478 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e7380-c9ef-40b9-8667-7369b035c598 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:26.913267 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e7380-c9ef-40b9-8667-7369b035c598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:26.915570 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e7380-c9ef-40b9-8667-7369b035c598 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:26.954677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:26.975268 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 893/1785] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:26 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:26.983045 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3fa844-0450-4848-a8cb-4162c1861c2b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:27.033193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3fa844-0450-4848-a8cb-4162c1861c2b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:27.083292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:27.192266 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522314f-17f9-4606-a997-4d5519234f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:27.193045 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522314f-17f9-4606-a997-4d5519234f1f admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:27.193379 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522314f-17f9-4606-a997-4d5519234f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:27.193656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522314f-17f9-4606-a997-4d5519234f1f admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:27.194310 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522314f-17f9-4606-a997-4d5519234f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:27.204842 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522314f-17f9-4606-a997-4d5519234f1f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:27.254316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 893/1786] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:26 2022] POST /identity/v3/users => generated 291 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:27.260969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e7380-c9ef-40b9-8667-7369b035c598 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:27.284546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e7380-c9ef-40b9-8667-7369b035c598 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:27.297443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b7106-f584-429e-910a-5f3e7890565d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:27.298696 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b7106-f584-429e-910a-5f3e7890565d admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:27.299283 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b7106-f584-429e-910a-5f3e7890565d admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:27.300311 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b7106-f584-429e-910a-5f3e7890565d admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:27.301455 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b7106-f584-429e-910a-5f3e7890565d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:27.304798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b7106-f584-429e-910a-5f3e7890565d admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:27.328144 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 894/1787] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:27.335043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522314f-17f9-4606-a997-4d5519234f1f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:27.347396 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522314f-17f9-4606-a997-4d5519234f1f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:27.355524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e46e1c-3fc0-4ec4-88cd-b3cab3e43792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:27.356416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e46e1c-3fc0-4ec4-88cd-b3cab3e43792 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:27.356877 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e46e1c-3fc0-4ec4-88cd-b3cab3e43792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:27.357530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e46e1c-3fc0-4ec4-88cd-b3cab3e43792 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:27.358419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e46e1c-3fc0-4ec4-88cd-b3cab3e43792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:27.360918 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e46e1c-3fc0-4ec4-88cd-b3cab3e43792 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:27.418014 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 894/1788] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:27.434948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b7106-f584-429e-910a-5f3e7890565d admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:27.452531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b7106-f584-429e-910a-5f3e7890565d admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:27.489942 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acef9ee-4e13-4210-9189-81a2df4ae9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:27.492300 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acef9ee-4e13-4210-9189-81a2df4ae9ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:27.496943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acef9ee-4e13-4210-9189-81a2df4ae9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:27.498080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acef9ee-4e13-4210-9189-81a2df4ae9ef admin admin] PATH_INFO: `/v3/projects/a67104040f6246e09b0cb9e574875e59/users/45aa825a872d43bbad41498e03ecb4cc/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:27.568799 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acef9ee-4e13-4210-9189-81a2df4ae9ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=a67104040f6246e09b0cb9e574875e59, user_id=45aa825a872d43bbad41498e03ecb4cc, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:27.580138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acef9ee-4e13-4210-9189-81a2df4ae9ef admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:27.618002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 895/1789] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:27 2022] PUT /identity/v3/projects/a67104040f6246e09b0cb9e574875e59/users/45aa825a872d43bbad41498e03ecb4cc/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:27.631282 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c71e6-7b94-41d4-8159-edd3983f79f1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:27.631757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c71e6-7b94-41d4-8159-edd3983f79f1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:27.632370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c71e6-7b94-41d4-8159-edd3983f79f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:27.698204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c74c71e6-7b94-41d4-8159-edd3983f79f1 None None] MFA Rules not processed for user `45aa825a872d43bbad41498e03ecb4cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:27.703079 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c74c71e6-7b94-41d4-8159-edd3983f79f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:27.814149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acef9ee-4e13-4210-9189-81a2df4ae9ef admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:27.817820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4acef9ee-4e13-4210-9189-81a2df4ae9ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:27.921359 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 895/1790] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:27.923222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c71e6-7b94-41d4-8159-edd3983f79f1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:27.981555 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acef9ee-4e13-4210-9189-81a2df4ae9ef admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:27.995096 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02701a-ce63-4c93-b57b-3df6d9d9d80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:27.995887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02701a-ce63-4c93-b57b-3df6d9d9d80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:27.996178 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02701a-ce63-4c93-b57b-3df6d9d9d80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:27.996457 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02701a-ce63-4c93-b57b-3df6d9d9d80a admin admin] PATH_INFO: `/v3/users/4235d35b5b8a4cf6a7055b8b81f3ad8b` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:27.997998 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02701a-ce63-4c93-b57b-3df6d9d9d80a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4235d35b5b8a4cf6a7055b8b81f3ad8b)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:28.000057 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02701a-ce63-4c93-b57b-3df6d9d9d80a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:28.031845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c71e6-7b94-41d4-8159-edd3983f79f1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:28.038109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8170f-39f7-4dda-8530-e856c3c20a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:28.038834 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8170f-39f7-4dda-8530-e856c3c20a73 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:28.039241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8170f-39f7-4dda-8530-e856c3c20a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:28.039584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8170f-39f7-4dda-8530-e856c3c20a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:28.040442 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8170f-39f7-4dda-8530-e856c3c20a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:28.044219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8170f-39f7-4dda-8530-e856c3c20a73 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:28.074323 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-bf02701a-ce63-4c93-b57b-3df6d9d9d80a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4235d35b5b8a4cf6a7055b8b81f3ad8b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:28.088218 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-bf02701a-ce63-4c93-b57b-3df6d9d9d80a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4235d35b5b8a4cf6a7055b8b81f3ad8b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:28.088971 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-bf02701a-ce63-4c93-b57b-3df6d9d9d80a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4235d35b5b8a4cf6a7055b8b81f3ad8b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:28.089983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-bf02701a-ce63-4c93-b57b-3df6d9d9d80a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4235d35b5b8a4cf6a7055b8b81f3ad8b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:28.099136 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-bf02701a-ce63-4c93-b57b-3df6d9d9d80a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4235d35b5b8a4cf6a7055b8b81f3ad8b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:28.099773 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 896/1791] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:27 2022] POST /identity/v3/projects => generated 379 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:28.105430 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8170f-39f7-4dda-8530-e856c3c20a73 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:28.121609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-09c8170f-39f7-4dda-8530-e856c3c20a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:28.123077 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 896/1792] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:27 2022] DELETE /identity/v3/users/4235d35b5b8a4cf6a7055b8b81f3ad8b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:45:28.127450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9692d-a563-4375-9497-6eef8fd06225 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:28.127705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9692d-a563-4375-9497-6eef8fd06225 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:28.127980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9692d-a563-4375-9497-6eef8fd06225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:28.148592 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-fdb9692d-a563-4375-9497-6eef8fd06225 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:28.149897 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-fdb9692d-a563-4375-9497-6eef8fd06225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:28.150118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:28.197380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:28.212056 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8170f-39f7-4dda-8530-e856c3c20a73 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:28.223771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 897/1793] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:45:28.236981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9692d-a563-4375-9497-6eef8fd06225 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:28.238865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-fdb9692d-a563-4375-9497-6eef8fd06225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:28.292494 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8c001-ebbb-4734-a049-01d6778dfed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:28.293243 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8c001-ebbb-4734-a049-01d6778dfed6 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:28.293531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8c001-ebbb-4734-a049-01d6778dfed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:28.293915 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8c001-ebbb-4734-a049-01d6778dfed6 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:28.294597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8c001-ebbb-4734-a049-01d6778dfed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:28.296682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8c001-ebbb-4734-a049-01d6778dfed6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:28.320431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9692d-a563-4375-9497-6eef8fd06225 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:28.322800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 897/1794] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:28 2022] POST /identity/v3/users => generated 296 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:45:28.324529 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289cd91-e6b3-4b26-ad16-a6d3defba551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:28.325222 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289cd91-e6b3-4b26-ad16-a6d3defba551 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:28.325512 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289cd91-e6b3-4b26-ad16-a6d3defba551 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:28.325815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289cd91-e6b3-4b26-ad16-a6d3defba551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:28.327838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8c001-ebbb-4734-a049-01d6778dfed6 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:28.339631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289cd91-e6b3-4b26-ad16-a6d3defba551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:28.340662 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8c001-ebbb-4734-a049-01d6778dfed6 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:28.341739 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289cd91-e6b3-4b26-ad16-a6d3defba551 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:28.347772 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5b9dd-c2f8-4c64-9fa4-afb4a9b5e16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:28.348495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5b9dd-c2f8-4c64-9fa4-afb4a9b5e16b admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:28.348778 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5b9dd-c2f8-4c64-9fa4-afb4a9b5e16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:28.349062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5b9dd-c2f8-4c64-9fa4-afb4a9b5e16b admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:28.349930 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5b9dd-c2f8-4c64-9fa4-afb4a9b5e16b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:28.351840 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5b9dd-c2f8-4c64-9fa4-afb4a9b5e16b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:28.358625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 898/1795] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:28.363600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 898/1796] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:28.375413 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289cd91-e6b3-4b26-ad16-a6d3defba551 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:28.387620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289cd91-e6b3-4b26-ad16-a6d3defba551 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:28.397688 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40dea9-d28b-4ab8-a2e9-1f6c540239ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:28.398575 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40dea9-d28b-4ab8-a2e9-1f6c540239ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:28.398862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40dea9-d28b-4ab8-a2e9-1f6c540239ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:28.399691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40dea9-d28b-4ab8-a2e9-1f6c540239ec admin admin] PATH_INFO: `/v3/projects/2165b6cfb594445cb37c8366222f2404/users/1b6a1f1c654c4cad9843e8b6fa42d3a4/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:28.433862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40dea9-d28b-4ab8-a2e9-1f6c540239ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=2165b6cfb594445cb37c8366222f2404, user_id=1b6a1f1c654c4cad9843e8b6fa42d3a4, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:28.437683 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40dea9-d28b-4ab8-a2e9-1f6c540239ec admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:28.486307 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 899/1797] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:28 2022] PUT /identity/v3/projects/2165b6cfb594445cb37c8366222f2404/users/1b6a1f1c654c4cad9843e8b6fa42d3a4/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:28.491022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5b9dd-c2f8-4c64-9fa4-afb4a9b5e16b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:28.558747 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5b9dd-c2f8-4c64-9fa4-afb4a9b5e16b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:28.656303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348441e-1df8-414b-b8f5-c9e00f38bb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:28.656692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348441e-1df8-414b-b8f5-c9e00f38bb0c admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:28.656963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348441e-1df8-414b-b8f5-c9e00f38bb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:28.657658 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348441e-1df8-414b-b8f5-c9e00f38bb0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:28.658461 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348441e-1df8-414b-b8f5-c9e00f38bb0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:28.660777 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348441e-1df8-414b-b8f5-c9e00f38bb0c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:28.675117 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 899/1798] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:28.681043 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40dea9-d28b-4ab8-a2e9-1f6c540239ec admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:28.695144 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40dea9-d28b-4ab8-a2e9-1f6c540239ec admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:28.700610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dee20b-8aca-47d8-9f7b-148396cea5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:28.701368 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dee20b-8aca-47d8-9f7b-148396cea5bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:28.702630 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dee20b-8aca-47d8-9f7b-148396cea5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:28.703033 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dee20b-8aca-47d8-9f7b-148396cea5bc admin admin] PATH_INFO: `/v3/projects/2165b6cfb594445cb37c8366222f2404/users/1b6a1f1c654c4cad9843e8b6fa42d3a4/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:28.706140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dee20b-8aca-47d8-9f7b-148396cea5bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2165b6cfb594445cb37c8366222f2404, user_id=1b6a1f1c654c4cad9843e8b6fa42d3a4, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:28.709239 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dee20b-8aca-47d8-9f7b-148396cea5bc admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:28.757423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 900/1799] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:28 2022] PUT /identity/v3/projects/2165b6cfb594445cb37c8366222f2404/users/1b6a1f1c654c4cad9843e8b6fa42d3a4/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:28.769814 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2d47d-caef-4bd8-8ae9-fc8ac7152abf None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:28.770541 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2d47d-caef-4bd8-8ae9-fc8ac7152abf None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:28.771004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2d47d-caef-4bd8-8ae9-fc8ac7152abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:28.822231 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-1ac2d47d-caef-4bd8-8ae9-fc8ac7152abf None None] MFA Rules not processed for user `1b6a1f1c654c4cad9843e8b6fa42d3a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:28.823790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1ac2d47d-caef-4bd8-8ae9-fc8ac7152abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:28.887341 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:28.952248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 900/1800] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:28.972301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dee20b-8aca-47d8-9f7b-148396cea5bc admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:28.975675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-e9dee20b-8aca-47d8-9f7b-148396cea5bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:29.130519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dee20b-8aca-47d8-9f7b-148396cea5bc admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:29.136436 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41584106-acbb-4a5e-94c6-d00a9e71cf3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:29.137173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41584106-acbb-4a5e-94c6-d00a9e71cf3b service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:29.137525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41584106-acbb-4a5e-94c6-d00a9e71cf3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:29.137898 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41584106-acbb-4a5e-94c6-d00a9e71cf3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:29.147781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2d47d-caef-4bd8-8ae9-fc8ac7152abf None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:29.151156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1ac2d47d-caef-4bd8-8ae9-fc8ac7152abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:29.151792 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41584106-acbb-4a5e-94c6-d00a9e71cf3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:29.153843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41584106-acbb-4a5e-94c6-d00a9e71cf3b service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:29.176784 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 901/1801] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:29.427544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2d47d-caef-4bd8-8ae9-fc8ac7152abf None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:29.459761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e75cd-8fc1-430e-bc2d-9d47572e9cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:29.460443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e75cd-8fc1-430e-bc2d-9d47572e9cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:29.460875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e75cd-8fc1-430e-bc2d-9d47572e9cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:29.462042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e75cd-8fc1-430e-bc2d-9d47572e9cec admin admin] PATH_INFO: `/v3/projects/8c72d43256114ca298ccbbd9a6da1eb1` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:29.465454 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e75cd-8fc1-430e-bc2d-9d47572e9cec admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c72d43256114ca298ccbbd9a6da1eb1)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:29.470057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e75cd-8fc1-430e-bc2d-9d47572e9cec admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:29.627089 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-543e75cd-8fc1-430e-bc2d-9d47572e9cec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c72d43256114ca298ccbbd9a6da1eb1', 'request_id': 'req-543e75cd-8fc1-430e-bc2d-9d47572e9cec'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:29.642131 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-543e75cd-8fc1-430e-bc2d-9d47572e9cec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c72d43256114ca298ccbbd9a6da1eb1', 'request_id': 'req-543e75cd-8fc1-430e-bc2d-9d47572e9cec'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:29.665387 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-543e75cd-8fc1-430e-bc2d-9d47572e9cec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c72d43256114ca298ccbbd9a6da1eb1', 'request_id': 'req-543e75cd-8fc1-430e-bc2d-9d47572e9cec'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:29.716964 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-543e75cd-8fc1-430e-bc2d-9d47572e9cec admin admin] The token cache is being invalidate because project 8c72d43256114ca298ccbbd9a6da1eb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:29.717412 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-543e75cd-8fc1-430e-bc2d-9d47572e9cec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:29.718048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-543e75cd-8fc1-430e-bc2d-9d47572e9cec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:29.719934 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 901/1802] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:29 2022] DELETE /identity/v3/projects/8c72d43256114ca298ccbbd9a6da1eb1 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:29.725576 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41584106-acbb-4a5e-94c6-d00a9e71cf3b service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:29.745979 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41584106-acbb-4a5e-94c6-d00a9e71cf3b service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:29.931151 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:29.947386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79897887-2805-4dc6-820d-8aa2f5f1d277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:29.948282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79897887-2805-4dc6-820d-8aa2f5f1d277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:29.948663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79897887-2805-4dc6-820d-8aa2f5f1d277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:29.949853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79897887-2805-4dc6-820d-8aa2f5f1d277 admin admin] PATH_INFO: `/v3/users/54aea2fcbd5e4dbea1efaa99f2af272d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:29.951671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79897887-2805-4dc6-820d-8aa2f5f1d277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54aea2fcbd5e4dbea1efaa99f2af272d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:29.954665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79897887-2805-4dc6-820d-8aa2f5f1d277 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:30.008742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e75cd-8fc1-430e-bc2d-9d47572e9cec admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:30.011175 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-543e75cd-8fc1-430e-bc2d-9d47572e9cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:30.069826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-79897887-2805-4dc6-820d-8aa2f5f1d277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54aea2fcbd5e4dbea1efaa99f2af272d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:30.090561 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-79897887-2805-4dc6-820d-8aa2f5f1d277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54aea2fcbd5e4dbea1efaa99f2af272d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:30.091385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-79897887-2805-4dc6-820d-8aa2f5f1d277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54aea2fcbd5e4dbea1efaa99f2af272d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:30.092187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-79897887-2805-4dc6-820d-8aa2f5f1d277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54aea2fcbd5e4dbea1efaa99f2af272d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:30.126458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-79897887-2805-4dc6-820d-8aa2f5f1d277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54aea2fcbd5e4dbea1efaa99f2af272d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:30.174250 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:30.193559 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e75cd-8fc1-430e-bc2d-9d47572e9cec admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:30.198895 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535e3fa-074c-44c8-8fad-307605974cca tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b6a1f1c654c4cad9843e8b6fa42d3a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2165b6cfb594445cb37c8366222f2404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:30.199145 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535e3fa-074c-44c8-8fad-307605974cca tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:30.199770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535e3fa-074c-44c8-8fad-307605974cca tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:30.199770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535e3fa-074c-44c8-8fad-307605974cca tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:30.200248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535e3fa-074c-44c8-8fad-307605974cca tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:45:30.200557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535e3fa-074c-44c8-8fad-307605974cca tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:30.201407 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 902/1803] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:29 2022] DELETE /identity/v3/users/54aea2fcbd5e4dbea1efaa99f2af272d => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:30.206834 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535e3fa-074c-44c8-8fad-307605974cca tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:30.224622 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 902/1804] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:45:30.230803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79897887-2805-4dc6-820d-8aa2f5f1d277 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:30.232445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-79897887-2805-4dc6-820d-8aa2f5f1d277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:30.373307 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79897887-2805-4dc6-820d-8aa2f5f1d277 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:30.380324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81dbdc8-a3c2-49c1-8834-b88babb84a7c tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b6a1f1c654c4cad9843e8b6fa42d3a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2165b6cfb594445cb37c8366222f2404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:30.380324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81dbdc8-a3c2-49c1-8834-b88babb84a7c tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:30.380785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81dbdc8-a3c2-49c1-8834-b88babb84a7c tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:30.382523 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81dbdc8-a3c2-49c1-8834-b88babb84a7c tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:30.382523 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81dbdc8-a3c2-49c1-8834-b88babb84a7c tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:30.384780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81dbdc8-a3c2-49c1-8834-b88babb84a7c tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:30.434285 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 903/1805] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:30 2022] POST /identity/v3/projects => generated 354 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:30.436699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535e3fa-074c-44c8-8fad-307605974cca tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:30.456716 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535e3fa-074c-44c8-8fad-307605974cca tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:30.583358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f6722-9f78-4e24-9913-0f9f9426c923 tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b6a1f1c654c4cad9843e8b6fa42d3a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2165b6cfb594445cb37c8366222f2404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:30.583994 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f6722-9f78-4e24-9913-0f9f9426c923 tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:30.584279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f6722-9f78-4e24-9913-0f9f9426c923 tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:30.584555 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f6722-9f78-4e24-9913-0f9f9426c923 tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:30.585410 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f6722-9f78-4e24-9913-0f9f9426c923 tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:45:30.586335 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f6722-9f78-4e24-9913-0f9f9426c923 tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:30.587969 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f6722-9f78-4e24-9913-0f9f9426c923 tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:30.607213 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 903/1806] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:45:30.677245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81dbdc8-a3c2-49c1-8834-b88babb84a7c tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:30.698023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81dbdc8-a3c2-49c1-8834-b88babb84a7c tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:30.705826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915490f-9846-4380-839b-36d5c619d1bd tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b6a1f1c654c4cad9843e8b6fa42d3a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2165b6cfb594445cb37c8366222f2404', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:30.708260 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915490f-9846-4380-839b-36d5c619d1bd tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:30.708641 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915490f-9846-4380-839b-36d5c619d1bd tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:30.708929 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915490f-9846-4380-839b-36d5c619d1bd tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] PATH_INFO: `/v3/projects/92cec61e31534d39873d6e8418f9b0c4` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:30.712950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915490f-9846-4380-839b-36d5c619d1bd tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] RBAC: Authorizing `identity:delete_project(project_id=92cec61e31534d39873d6e8418f9b0c4)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:30.715696 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915490f-9846-4380-839b-36d5c619d1bd tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:30.796611 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-b915490f-9846-4380-839b-36d5c619d1bd tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92cec61e31534d39873d6e8418f9b0c4', 'request_id': 'req-b915490f-9846-4380-839b-36d5c619d1bd'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:30.799392 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:30.806004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-b915490f-9846-4380-839b-36d5c619d1bd tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92cec61e31534d39873d6e8418f9b0c4', 'request_id': 'req-b915490f-9846-4380-839b-36d5c619d1bd'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:30.821318 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-b915490f-9846-4380-839b-36d5c619d1bd tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92cec61e31534d39873d6e8418f9b0c4', 'request_id': 'req-b915490f-9846-4380-839b-36d5c619d1bd'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:30.839821 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-b915490f-9846-4380-839b-36d5c619d1bd tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] The token cache is being invalidate because project 92cec61e31534d39873d6e8418f9b0c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:30.840206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-b915490f-9846-4380-839b-36d5c619d1bd tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:30.841686 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-b915490f-9846-4380-839b-36d5c619d1bd tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:30.843654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 904/1807] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:30 2022] DELETE /identity/v3/projects/92cec61e31534d39873d6e8418f9b0c4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:31.204035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f6722-9f78-4e24-9913-0f9f9426c923 tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:31.206303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ec3f6722-9f78-4e24-9913-0f9f9426c923 tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:31.341011 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb2259-806f-4ec5-b339-0ab3dbe096ae None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:31.341334 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb2259-806f-4ec5-b339-0ab3dbe096ae None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:31.341615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb2259-806f-4ec5-b339-0ab3dbe096ae None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:31.343198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 905/1808] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 26 05:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:45:31.344442 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f6722-9f78-4e24-9913-0f9f9426c923 tempest-VolumeQuotaClassesTest-166024981 tempest-VolumeQuotaClassesTest-166024981-project-admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:31.351950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d0b38-695e-4123-b6c3-80c4b22e3338 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:31.352241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d0b38-695e-4123-b6c3-80c4b22e3338 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:31.352379 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d0b38-695e-4123-b6c3-80c4b22e3338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:31.389597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d93d0b38-695e-4123-b6c3-80c4b22e3338 None None] MFA Rules not processed for user `67fcd65eeb994b2381b83480e030f2ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:31.390967 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d93d0b38-695e-4123-b6c3-80c4b22e3338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:31.525977 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d93f8a-058b-409b-8b3d-7462b9b2ebef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:31.527815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d93f8a-058b-409b-8b3d-7462b9b2ebef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:31.528625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d93f8a-058b-409b-8b3d-7462b9b2ebef admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:31.529092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d93f8a-058b-409b-8b3d-7462b9b2ebef admin admin] PATH_INFO: `/v3/users/45aa825a872d43bbad41498e03ecb4cc` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:31.531413 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d93f8a-058b-409b-8b3d-7462b9b2ebef admin admin] RBAC: Authorizing `identity:delete_user(user_id=45aa825a872d43bbad41498e03ecb4cc)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:31.532080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 906/1809] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 26 05:45:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:45:31.533809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d0b38-695e-4123-b6c3-80c4b22e3338 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:31.535518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d93f8a-058b-409b-8b3d-7462b9b2ebef admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:31.536754 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d93d0b38-695e-4123-b6c3-80c4b22e3338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:31.564735 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d0b38-695e-4123-b6c3-80c4b22e3338 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:31.588065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca58c6-6b5a-408d-b329-8c764bed51f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:31.589446 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca58c6-6b5a-408d-b329-8c764bed51f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:31.590035 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca58c6-6b5a-408d-b329-8c764bed51f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:31.591716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca58c6-6b5a-408d-b329-8c764bed51f7 admin admin] PATH_INFO: `/v3/projects/aa293b1ed6674ae48499044c6850081c` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:31.598999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca58c6-6b5a-408d-b329-8c764bed51f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa293b1ed6674ae48499044c6850081c)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:31.607204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca58c6-6b5a-408d-b329-8c764bed51f7 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:31.677490 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:31.717017 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-42d93f8a-058b-409b-8b3d-7462b9b2ebef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45aa825a872d43bbad41498e03ecb4cc'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:31.732865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-b1ca58c6-6b5a-408d-b329-8c764bed51f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa293b1ed6674ae48499044c6850081c', 'request_id': 'req-b1ca58c6-6b5a-408d-b329-8c764bed51f7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:31.740385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-b1ca58c6-6b5a-408d-b329-8c764bed51f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa293b1ed6674ae48499044c6850081c', 'request_id': 'req-b1ca58c6-6b5a-408d-b329-8c764bed51f7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:31.749253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-42d93f8a-058b-409b-8b3d-7462b9b2ebef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45aa825a872d43bbad41498e03ecb4cc'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:31.753004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-b1ca58c6-6b5a-408d-b329-8c764bed51f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa293b1ed6674ae48499044c6850081c', 'request_id': 'req-b1ca58c6-6b5a-408d-b329-8c764bed51f7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:31.753301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-42d93f8a-058b-409b-8b3d-7462b9b2ebef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45aa825a872d43bbad41498e03ecb4cc'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:31.758884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-42d93f8a-058b-409b-8b3d-7462b9b2ebef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45aa825a872d43bbad41498e03ecb4cc'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:31.777044 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-b1ca58c6-6b5a-408d-b329-8c764bed51f7 admin admin] The token cache is being invalidate because project aa293b1ed6674ae48499044c6850081c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:31.777960 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-b1ca58c6-6b5a-408d-b329-8c764bed51f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:31.779566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-b1ca58c6-6b5a-408d-b329-8c764bed51f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:31.782707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 907/1810] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:31 2022] DELETE /identity/v3/projects/aa293b1ed6674ae48499044c6850081c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:31.785601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-42d93f8a-058b-409b-8b3d-7462b9b2ebef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45aa825a872d43bbad41498e03ecb4cc'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:31.788854 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b1457-e525-4ebe-9f07-6919e1c64684 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:31.789198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b1457-e525-4ebe-9f07-6919e1c64684 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:31.789449 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b1457-e525-4ebe-9f07-6919e1c64684 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:31.790887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 908/1811] 10.222.0.57 () {54 vars in 961 bytes} [Mon Sep 26 05:45:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:45:31.792196 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b5466-13ec-4d43-af23-98f7c2d6ddba None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:31.792485 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b5466-13ec-4d43-af23-98f7c2d6ddba None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:31.793762 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b5466-13ec-4d43-af23-98f7c2d6ddba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:31.814605 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-7c8b5466-13ec-4d43-af23-98f7c2d6ddba None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:31.816075 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7c8b5466-13ec-4d43-af23-98f7c2d6ddba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:31.823819 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 904/1812] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:31 2022] DELETE /identity/v3/users/45aa825a872d43bbad41498e03ecb4cc => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:45:31.825679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546295e5-4184-42b5-843e-9c29673ff7cb None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:31.825970 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546295e5-4184-42b5-843e-9c29673ff7cb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:31.826246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546295e5-4184-42b5-843e-9c29673ff7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:31.850982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-546295e5-4184-42b5-843e-9c29673ff7cb None None] MFA Rules not processed for user `67fcd65eeb994b2381b83480e030f2ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:31.852245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-546295e5-4184-42b5-843e-9c29673ff7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:31.890637 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:31.902009 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 909/1813] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:31.910952 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb78dd-7214-4574-ac33-ec51e75cc45e None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:31.911758 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb78dd-7214-4574-ac33-ec51e75cc45e None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:31.912102 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb78dd-7214-4574-ac33-ec51e75cc45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:31.937669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-43eb78dd-7214-4574-ac33-ec51e75cc45e None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:31.939675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-43eb78dd-7214-4574-ac33-ec51e75cc45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:31.992299 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 905/1814] 10.222.0.57 () {58 vars in 1058 bytes} [Mon Sep 26 05:45:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:32.000669 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccf6c0-fdc5-4292-bc89-1a755d72ab55 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:32.001514 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccf6c0-fdc5-4292-bc89-1a755d72ab55 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:32.002147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccf6c0-fdc5-4292-bc89-1a755d72ab55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:32.054326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-acccf6c0-fdc5-4292-bc89-1a755d72ab55 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:32.057393 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-acccf6c0-fdc5-4292-bc89-1a755d72ab55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:32.091622 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 910/1815] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:32.093008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb78dd-7214-4574-ac33-ec51e75cc45e None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:32.115873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb78dd-7214-4574-ac33-ec51e75cc45e None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:32.129356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cfb8df-f90c-4f8e-9b94-305b6bc38a9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:32.130185 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfb8df-f90c-4f8e-9b94-305b6bc38a9b service placement] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:32.130569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfb8df-f90c-4f8e-9b94-305b6bc38a9b service placement] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:32.130974 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfb8df-f90c-4f8e-9b94-305b6bc38a9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:32.133284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-76cfb8df-f90c-4f8e-9b94-305b6bc38a9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:32.154695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cfb8df-f90c-4f8e-9b94-305b6bc38a9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:32.159779 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cfb8df-f90c-4f8e-9b94-305b6bc38a9b service placement] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:32.173063 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 906/1816] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:32.173063 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccf6c0-fdc5-4292-bc89-1a755d72ab55 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:32.174672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-acccf6c0-fdc5-4292-bc89-1a755d72ab55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:32.183137 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 911/1817] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:32.185002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cfb8df-f90c-4f8e-9b94-305b6bc38a9b service placement] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:32.205024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cfb8df-f90c-4f8e-9b94-305b6bc38a9b service placement] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:32.223377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7ecb9-5ae4-4c03-a5db-1496a93d542d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:32.224259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7ecb9-5ae4-4c03-a5db-1496a93d542d admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:32.225081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7ecb9-5ae4-4c03-a5db-1496a93d542d admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:32.225602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7ecb9-5ae4-4c03-a5db-1496a93d542d admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:32.226510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7ecb9-5ae4-4c03-a5db-1496a93d542d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:45:32.227420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7ecb9-5ae4-4c03-a5db-1496a93d542d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:32.237233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7ecb9-5ae4-4c03-a5db-1496a93d542d admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:32.269191 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 912/1818] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:45:32.274788 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7ecb9-5ae4-4c03-a5db-1496a93d542d admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:32.276725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:32.289710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccf6c0-fdc5-4292-bc89-1a755d72ab55 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:32.290548 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7ecb9-5ae4-4c03-a5db-1496a93d542d admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:32.296500 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ac1e6-e247-49e5-9507-ce4c1753d136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:32.297928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29708e3c-87a4-4aad-8bce-5f5d618eb4c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:32.298090 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ac1e6-e247-49e5-9507-ce4c1753d136 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:32.298090 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ac1e6-e247-49e5-9507-ce4c1753d136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:32.298090 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ac1e6-e247-49e5-9507-ce4c1753d136 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:32.298285 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29708e3c-87a4-4aad-8bce-5f5d618eb4c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:32.298900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ac1e6-e247-49e5-9507-ce4c1753d136 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:32.299010 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29708e3c-87a4-4aad-8bce-5f5d618eb4c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:32.299449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29708e3c-87a4-4aad-8bce-5f5d618eb4c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:32.300978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ac1e6-e247-49e5-9507-ce4c1753d136 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:32.313490 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29708e3c-87a4-4aad-8bce-5f5d618eb4c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:32.317781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29708e3c-87a4-4aad-8bce-5f5d618eb4c0 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:32.330663 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 907/1819] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:32.340411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 913/1820] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:32 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:32.342183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29708e3c-87a4-4aad-8bce-5f5d618eb4c0 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:32.356931 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29708e3c-87a4-4aad-8bce-5f5d618eb4c0 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:32.448095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c73633-8db0-49cf-a452-aa8e3cad8bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:32.448769 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c73633-8db0-49cf-a452-aa8e3cad8bab admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:32.450914 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c73633-8db0-49cf-a452-aa8e3cad8bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:32.451249 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c73633-8db0-49cf-a452-aa8e3cad8bab admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:32.451895 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c73633-8db0-49cf-a452-aa8e3cad8bab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:32.454104 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c73633-8db0-49cf-a452-aa8e3cad8bab admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:32.486674 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ac1e6-e247-49e5-9507-ce4c1753d136 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:32.491383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-412ac1e6-e247-49e5-9507-ce4c1753d136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:32.492077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 908/1821] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:32 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:32.493449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c73633-8db0-49cf-a452-aa8e3cad8bab admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:32.508070 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c73633-8db0-49cf-a452-aa8e3cad8bab admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:32.513804 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef2aaa-902b-4597-a941-77df7e8b0259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:32.514721 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef2aaa-902b-4597-a941-77df7e8b0259 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:32.515012 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef2aaa-902b-4597-a941-77df7e8b0259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:32.515279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef2aaa-902b-4597-a941-77df7e8b0259 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:32.515767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ac1e6-e247-49e5-9507-ce4c1753d136 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:32.516130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef2aaa-902b-4597-a941-77df7e8b0259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:32.518136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef2aaa-902b-4597-a941-77df7e8b0259 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:32.524141 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a781e602-f237-4875-9bd7-54c5bb0a4cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:32.524751 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781e602-f237-4875-9bd7-54c5bb0a4cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:32.525030 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781e602-f237-4875-9bd7-54c5bb0a4cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:32.528077 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781e602-f237-4875-9bd7-54c5bb0a4cb5 admin admin] PATH_INFO: `/v3/users/b7dd624acd6f43dbba714e8bb10fa95a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:32.528606 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781e602-f237-4875-9bd7-54c5bb0a4cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7dd624acd6f43dbba714e8bb10fa95a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:32.531157 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a781e602-f237-4875-9bd7-54c5bb0a4cb5 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:32.537461 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 909/1822] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:32.540567 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef2aaa-902b-4597-a941-77df7e8b0259 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:32.576431 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef2aaa-902b-4597-a941-77df7e8b0259 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:32.584104 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695e3bd-04d6-410a-bf65-81a7fc95a610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:32.585039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695e3bd-04d6-410a-bf65-81a7fc95a610 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:32.585535 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695e3bd-04d6-410a-bf65-81a7fc95a610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:32.585930 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695e3bd-04d6-410a-bf65-81a7fc95a610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:32.587409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695e3bd-04d6-410a-bf65-81a7fc95a610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:32.590029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695e3bd-04d6-410a-bf65-81a7fc95a610 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:32.629870 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a781e602-f237-4875-9bd7-54c5bb0a4cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7dd624acd6f43dbba714e8bb10fa95a'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:32.637810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 910/1823] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:32.643761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695e3bd-04d6-410a-bf65-81a7fc95a610 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:32.653333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a781e602-f237-4875-9bd7-54c5bb0a4cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7dd624acd6f43dbba714e8bb10fa95a'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:32.654171 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a781e602-f237-4875-9bd7-54c5bb0a4cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7dd624acd6f43dbba714e8bb10fa95a'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:32.655078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a781e602-f237-4875-9bd7-54c5bb0a4cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7dd624acd6f43dbba714e8bb10fa95a'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:32.676202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695e3bd-04d6-410a-bf65-81a7fc95a610 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:32.680202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a781e602-f237-4875-9bd7-54c5bb0a4cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7dd624acd6f43dbba714e8bb10fa95a'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:32.686984 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b25fd-1862-432f-9b66-e28768f5ef22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:32.688921 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b25fd-1862-432f-9b66-e28768f5ef22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:32.689826 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b25fd-1862-432f-9b66-e28768f5ef22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:32.690450 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b25fd-1862-432f-9b66-e28768f5ef22 admin admin] PATH_INFO: `/v3/projects/edced8c05f4a4e8cb85b44c49505a829/users/c8681b583db24403aa873f60c5f30a2f/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:32.715927 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 914/1824] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:32 2022] DELETE /identity/v3/users/b7dd624acd6f43dbba714e8bb10fa95a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:45:32.729284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b25fd-1862-432f-9b66-e28768f5ef22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edced8c05f4a4e8cb85b44c49505a829, user_id=c8681b583db24403aa873f60c5f30a2f, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:32.732352 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b25fd-1862-432f-9b66-e28768f5ef22 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:32.752483 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 911/1825] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:32 2022] PUT /identity/v3/projects/edced8c05f4a4e8cb85b44c49505a829/users/c8681b583db24403aa873f60c5f30a2f/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:32.776347 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b4e6a-e4d1-4928-85e9-8697d18c17a4 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:32.776674 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b4e6a-e4d1-4928-85e9-8697d18c17a4 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:32.777525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b4e6a-e4d1-4928-85e9-8697d18c17a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:32.864316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-c45b4e6a-e4d1-4928-85e9-8697d18c17a4 None None] MFA Rules not processed for user `c8681b583db24403aa873f60c5f30a2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:32.866171 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c45b4e6a-e4d1-4928-85e9-8697d18c17a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:32.881014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:32.947552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 915/1826] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:45:32.955200 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b25fd-1862-432f-9b66-e28768f5ef22 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:32.958111 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-788b25fd-1862-432f-9b66-e28768f5ef22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:32.966892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b4e6a-e4d1-4928-85e9-8697d18c17a4 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:32.971185 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c45b4e6a-e4d1-4928-85e9-8697d18c17a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:33.112801 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b4e6a-e4d1-4928-85e9-8697d18c17a4 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:33.113645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b25fd-1862-432f-9b66-e28768f5ef22 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:33.119780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feeac35-5804-4145-9b3f-55a3b29b4f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:33.121244 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feeac35-5804-4145-9b3f-55a3b29b4f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:33.121244 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feeac35-5804-4145-9b3f-55a3b29b4f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:33.121244 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feeac35-5804-4145-9b3f-55a3b29b4f64 admin admin] PATH_INFO: `/v3/projects/a67104040f6246e09b0cb9e574875e59` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:33.122761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feeac35-5804-4145-9b3f-55a3b29b4f64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a67104040f6246e09b0cb9e574875e59)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:33.124858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feeac35-5804-4145-9b3f-55a3b29b4f64 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:33.137192 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdd82b-a089-4ae1-9859-398d4edc0d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:33.140590 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdd82b-a089-4ae1-9859-398d4edc0d13 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:33.141283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdd82b-a089-4ae1-9859-398d4edc0d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:33.141854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdd82b-a089-4ae1-9859-398d4edc0d13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:33.143817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdd82b-a089-4ae1-9859-398d4edc0d13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:33.151636 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdd82b-a089-4ae1-9859-398d4edc0d13 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:33.214505 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1feeac35-5804-4145-9b3f-55a3b29b4f64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a67104040f6246e09b0cb9e574875e59', 'request_id': 'req-1feeac35-5804-4145-9b3f-55a3b29b4f64'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.214704 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 912/1827] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:32 2022] POST /identity/v3/projects => generated 387 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:33.215522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdd82b-a089-4ae1-9859-398d4edc0d13 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:33.217697 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-54bdd82b-a089-4ae1-9859-398d4edc0d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:33.235682 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1feeac35-5804-4145-9b3f-55a3b29b4f64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a67104040f6246e09b0cb9e574875e59', 'request_id': 'req-1feeac35-5804-4145-9b3f-55a3b29b4f64'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.244776 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:33.250159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1feeac35-5804-4145-9b3f-55a3b29b4f64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a67104040f6246e09b0cb9e574875e59', 'request_id': 'req-1feeac35-5804-4145-9b3f-55a3b29b4f64'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.276630 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1feeac35-5804-4145-9b3f-55a3b29b4f64 admin admin] The token cache is being invalidate because project a67104040f6246e09b0cb9e574875e59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:33.277460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1feeac35-5804-4145-9b3f-55a3b29b4f64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.280066 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1feeac35-5804-4145-9b3f-55a3b29b4f64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.283043 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 916/1828] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:32 2022] DELETE /identity/v3/projects/a67104040f6246e09b0cb9e574875e59 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:33.284724 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feeac35-5804-4145-9b3f-55a3b29b4f64 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:33.289106 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-1feeac35-5804-4145-9b3f-55a3b29b4f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:33.317383 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdd82b-a089-4ae1-9859-398d4edc0d13 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:33.325197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:33.326252 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:33.326252 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:33.326571 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b admin admin] PATH_INFO: `/v3/projects/ee2bc9500eb9402f880f67c361e37fbb` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:33.327820 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee2bc9500eb9402f880f67c361e37fbb)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:33.330188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:33.385332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:33.399607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee2bc9500eb9402f880f67c361e37fbb', 'request_id': 'req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.407954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee2bc9500eb9402f880f67c361e37fbb', 'request_id': 'req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.419585 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee2bc9500eb9402f880f67c361e37fbb', 'request_id': 'req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.419894 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feeac35-5804-4145-9b3f-55a3b29b4f64 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:33.437376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b admin admin] The token cache is being invalidate because project ee2bc9500eb9402f880f67c361e37fbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:33.437671 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.438464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.440230 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 913/1829] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:33 2022] DELETE /identity/v3/projects/ee2bc9500eb9402f880f67c361e37fbb => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:33.441017 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:33.442600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:33.522085 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0f985-efbe-48cb-a657-aaa50245d746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:33.523118 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0f985-efbe-48cb-a657-aaa50245d746 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:33.523491 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0f985-efbe-48cb-a657-aaa50245d746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:33.523949 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0f985-efbe-48cb-a657-aaa50245d746 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:33.525033 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0f985-efbe-48cb-a657-aaa50245d746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:33.528808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0f985-efbe-48cb-a657-aaa50245d746 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:33.543767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c5dc0-dc8d-458e-aaf1-e6f0d181fe2b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:33.551466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9f5f1-568d-4458-a6fc-9c609a84df15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:33.552699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9f5f1-568d-4458-a6fc-9c609a84df15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:33.553714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9f5f1-568d-4458-a6fc-9c609a84df15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:33.554105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9f5f1-568d-4458-a6fc-9c609a84df15 admin admin] PATH_INFO: `/v3/users/1b6a1f1c654c4cad9843e8b6fa42d3a4` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:33.556038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9f5f1-568d-4458-a6fc-9c609a84df15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b6a1f1c654c4cad9843e8b6fa42d3a4)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:33.559857 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9f5f1-568d-4458-a6fc-9c609a84df15 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:33.565651 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 917/1830] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:33 2022] POST /identity/v3/users => generated 300 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:33.567905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0f985-efbe-48cb-a657-aaa50245d746 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:33.570964 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-94b0f985-efbe-48cb-a657-aaa50245d746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:33.590365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0f985-efbe-48cb-a657-aaa50245d746 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:33.598324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdac9b0-57a1-4045-af41-5e0ac8cd5e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:33.599746 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdac9b0-57a1-4045-af41-5e0ac8cd5e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:33.600404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdac9b0-57a1-4045-af41-5e0ac8cd5e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:33.600515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdac9b0-57a1-4045-af41-5e0ac8cd5e38 admin admin] PATH_INFO: `/v3/users/dce6e36226d644e4a399aaff7d15ce03` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:33.603352 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdac9b0-57a1-4045-af41-5e0ac8cd5e38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dce6e36226d644e4a399aaff7d15ce03)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:33.607969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdac9b0-57a1-4045-af41-5e0ac8cd5e38 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:33.651895 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-afa9f5f1-568d-4458-a6fc-9c609a84df15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b6a1f1c654c4cad9843e8b6fa42d3a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.656129 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:33.667216 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-afa9f5f1-568d-4458-a6fc-9c609a84df15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b6a1f1c654c4cad9843e8b6fa42d3a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.668071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-afa9f5f1-568d-4458-a6fc-9c609a84df15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b6a1f1c654c4cad9843e8b6fa42d3a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.669447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-afa9f5f1-568d-4458-a6fc-9c609a84df15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b6a1f1c654c4cad9843e8b6fa42d3a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.705425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-afa9f5f1-568d-4458-a6fc-9c609a84df15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b6a1f1c654c4cad9843e8b6fa42d3a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.710839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6fdac9b0-57a1-4045-af41-5e0ac8cd5e38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dce6e36226d644e4a399aaff7d15ce03'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.722480 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6fdac9b0-57a1-4045-af41-5e0ac8cd5e38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dce6e36226d644e4a399aaff7d15ce03'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.724001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6fdac9b0-57a1-4045-af41-5e0ac8cd5e38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dce6e36226d644e4a399aaff7d15ce03'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.724969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6fdac9b0-57a1-4045-af41-5e0ac8cd5e38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dce6e36226d644e4a399aaff7d15ce03'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.730054 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 914/1831] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:33 2022] DELETE /identity/v3/users/1b6a1f1c654c4cad9843e8b6fa42d3a4 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:33.731653 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9f5f1-568d-4458-a6fc-9c609a84df15 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:33.734209 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-afa9f5f1-568d-4458-a6fc-9c609a84df15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:33.742181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6fdac9b0-57a1-4045-af41-5e0ac8cd5e38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dce6e36226d644e4a399aaff7d15ce03'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:33.771751 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 918/1832] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:33 2022] DELETE /identity/v3/users/dce6e36226d644e4a399aaff7d15ce03 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:33.863469 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9f5f1-568d-4458-a6fc-9c609a84df15 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:33.965000 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fe28b-fc03-4800-b707-fab9602c7c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:33.966039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fe28b-fc03-4800-b707-fab9602c7c86 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:33.966516 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fe28b-fc03-4800-b707-fab9602c7c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:33.966989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14fe28b-fc03-4800-b707-fab9602c7c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:33.967948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fe28b-fc03-4800-b707-fab9602c7c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:33.970239 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14fe28b-fc03-4800-b707-fab9602c7c86 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:33.988814 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 915/1833] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:33.994380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdac9b0-57a1-4045-af41-5e0ac8cd5e38 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:33.997563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6fdac9b0-57a1-4045-af41-5e0ac8cd5e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:34.018225 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdac9b0-57a1-4045-af41-5e0ac8cd5e38 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:34.025457 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7ea50-049e-4490-8e15-1ebe82881e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:34.026582 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7ea50-049e-4490-8e15-1ebe82881e9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:34.026967 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7ea50-049e-4490-8e15-1ebe82881e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:34.027421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7ea50-049e-4490-8e15-1ebe82881e9d admin admin] PATH_INFO: `/v3/projects/ea7c92c066e548fe9444295d11dc22fc/users/990740b9f848425cb1dac419b6a4e2a3/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:34.058443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7ea50-049e-4490-8e15-1ebe82881e9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea7c92c066e548fe9444295d11dc22fc, user_id=990740b9f848425cb1dac419b6a4e2a3, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:34.067044 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7ea50-049e-4490-8e15-1ebe82881e9d admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:34.093417 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 919/1834] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:33 2022] PUT /identity/v3/projects/ea7c92c066e548fe9444295d11dc22fc/users/990740b9f848425cb1dac419b6a4e2a3/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:45:34.099112 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fe28b-fc03-4800-b707-fab9602c7c86 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:34.114831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14fe28b-fc03-4800-b707-fab9602c7c86 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:34.222036 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c268d0-b44f-4270-bd16-fc679fe641ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:34.222982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c268d0-b44f-4270-bd16-fc679fe641ee admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:34.223425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c268d0-b44f-4270-bd16-fc679fe641ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:34.223806 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c268d0-b44f-4270-bd16-fc679fe641ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:34.225758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c268d0-b44f-4270-bd16-fc679fe641ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:34.226491 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c268d0-b44f-4270-bd16-fc679fe641ee admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:34.252389 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 916/1835] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:34.256643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7ea50-049e-4490-8e15-1ebe82881e9d admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:34.270572 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7ea50-049e-4490-8e15-1ebe82881e9d admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:34.283895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94325752-f63b-46ca-9bec-947fb9030111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:34.283895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94325752-f63b-46ca-9bec-947fb9030111 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:34.283895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94325752-f63b-46ca-9bec-947fb9030111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:34.283895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94325752-f63b-46ca-9bec-947fb9030111 admin admin] PATH_INFO: `/v3/projects/ea7c92c066e548fe9444295d11dc22fc/users/990740b9f848425cb1dac419b6a4e2a3/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:34.288870 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94325752-f63b-46ca-9bec-947fb9030111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea7c92c066e548fe9444295d11dc22fc, user_id=990740b9f848425cb1dac419b6a4e2a3, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:34.294661 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94325752-f63b-46ca-9bec-947fb9030111 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:34.311843 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c268d0-b44f-4270-bd16-fc679fe641ee admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:34.315838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-34c268d0-b44f-4270-bd16-fc679fe641ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:34.329693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 920/1836] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:34 2022] PUT /identity/v3/projects/ea7c92c066e548fe9444295d11dc22fc/users/990740b9f848425cb1dac419b6a4e2a3/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:45:34.341669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6feec-faad-4ede-9fc9-c0d5d613b9da None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:34.342025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6feec-faad-4ede-9fc9-c0d5d613b9da None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:34.342368 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6feec-faad-4ede-9fc9-c0d5d613b9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:34.366931 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c268d0-b44f-4270-bd16-fc679fe641ee admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:34.373304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:34.466536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-f1b6feec-faad-4ede-9fc9-c0d5d613b9da None None] MFA Rules not processed for user `990740b9f848425cb1dac419b6a4e2a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:34.467470 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d7d29-1364-4b9a-a74b-dd9f09f252af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:34.468108 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f1b6feec-faad-4ede-9fc9-c0d5d613b9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:34.468456 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d7d29-1364-4b9a-a74b-dd9f09f252af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:34.469058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d7d29-1364-4b9a-a74b-dd9f09f252af admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:34.469508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d7d29-1364-4b9a-a74b-dd9f09f252af admin admin] PATH_INFO: `/v3/projects/7260bbfc59c640458e83ed0c0cdbfd5d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:34.471132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d7d29-1364-4b9a-a74b-dd9f09f252af admin admin] RBAC: Authorizing `identity:delete_project(project_id=7260bbfc59c640458e83ed0c0cdbfd5d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:34.473500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d7d29-1364-4b9a-a74b-dd9f09f252af admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:34.480348 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:34.546707 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b27d7d29-1364-4b9a-a74b-dd9f09f252af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7260bbfc59c640458e83ed0c0cdbfd5d', 'request_id': 'req-b27d7d29-1364-4b9a-a74b-dd9f09f252af'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:34.558735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b27d7d29-1364-4b9a-a74b-dd9f09f252af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7260bbfc59c640458e83ed0c0cdbfd5d', 'request_id': 'req-b27d7d29-1364-4b9a-a74b-dd9f09f252af'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:34.570259 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b27d7d29-1364-4b9a-a74b-dd9f09f252af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7260bbfc59c640458e83ed0c0cdbfd5d', 'request_id': 'req-b27d7d29-1364-4b9a-a74b-dd9f09f252af'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:34.588813 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b27d7d29-1364-4b9a-a74b-dd9f09f252af admin admin] The token cache is being invalidate because project 7260bbfc59c640458e83ed0c0cdbfd5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:34.589548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b27d7d29-1364-4b9a-a74b-dd9f09f252af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:34.590737 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b27d7d29-1364-4b9a-a74b-dd9f09f252af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:34.593195 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 917/1837] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:34 2022] DELETE /identity/v3/projects/7260bbfc59c640458e83ed0c0cdbfd5d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:34.594474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d7d29-1364-4b9a-a74b-dd9f09f252af admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:34.596872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b27d7d29-1364-4b9a-a74b-dd9f09f252af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:34.631591 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 921/1838] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:34 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:34.634271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c125fc-0b9b-4697-b6cd-5afd8ad4ec19 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:34.634855 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c125fc-0b9b-4697-b6cd-5afd8ad4ec19 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:34.635994 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c125fc-0b9b-4697-b6cd-5afd8ad4ec19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:34.680065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-75c125fc-0b9b-4697-b6cd-5afd8ad4ec19 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:34.682409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-75c125fc-0b9b-4697-b6cd-5afd8ad4ec19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:34.685071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d7d29-1364-4b9a-a74b-dd9f09f252af admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:34.691396 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd31351-fe92-4213-b26f-9de5a8cc12ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:34.692331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd31351-fe92-4213-b26f-9de5a8cc12ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:34.692847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd31351-fe92-4213-b26f-9de5a8cc12ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:34.693419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd31351-fe92-4213-b26f-9de5a8cc12ee admin admin] PATH_INFO: `/v3/projects/2165b6cfb594445cb37c8366222f2404` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:34.694778 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd31351-fe92-4213-b26f-9de5a8cc12ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=2165b6cfb594445cb37c8366222f2404)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:34.697892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd31351-fe92-4213-b26f-9de5a8cc12ee admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:34.708746 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 922/1839] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:34.708746 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c125fc-0b9b-4697-b6cd-5afd8ad4ec19 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:34.708746 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-75c125fc-0b9b-4697-b6cd-5afd8ad4ec19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:34.812814 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:34.816228 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c125fc-0b9b-4697-b6cd-5afd8ad4ec19 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:34.831977 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0cd31351-fe92-4213-b26f-9de5a8cc12ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2165b6cfb594445cb37c8366222f2404', 'request_id': 'req-0cd31351-fe92-4213-b26f-9de5a8cc12ee'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:34.844542 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0cd31351-fe92-4213-b26f-9de5a8cc12ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2165b6cfb594445cb37c8366222f2404', 'request_id': 'req-0cd31351-fe92-4213-b26f-9de5a8cc12ee'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:34.858669 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0cd31351-fe92-4213-b26f-9de5a8cc12ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2165b6cfb594445cb37c8366222f2404', 'request_id': 'req-0cd31351-fe92-4213-b26f-9de5a8cc12ee'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:34.876794 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0cd31351-fe92-4213-b26f-9de5a8cc12ee admin admin] The token cache is being invalidate because project 2165b6cfb594445cb37c8366222f2404 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:34.877241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0cd31351-fe92-4213-b26f-9de5a8cc12ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:34.878086 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0cd31351-fe92-4213-b26f-9de5a8cc12ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:34.882031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 918/1840] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:34 2022] DELETE /identity/v3/projects/2165b6cfb594445cb37c8366222f2404 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:34.883526 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d1f2a-b538-4a74-bb56-79a13a821de4 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:34.883899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d1f2a-b538-4a74-bb56-79a13a821de4 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:34.884202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d1f2a-b538-4a74-bb56-79a13a821de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:34.902376 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0df4a-a925-4436-9601-a4d4d2eabea6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:34.903045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0df4a-a925-4436-9601-a4d4d2eabea6 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:34.903442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0df4a-a925-4436-9601-a4d4d2eabea6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:34.903720 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0df4a-a925-4436-9601-a4d4d2eabea6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:34.906246 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-8ed0df4a-a925-4436-9601-a4d4d2eabea6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:34.910912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-bf7d1f2a-b538-4a74-bb56-79a13a821de4 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:34.912628 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bf7d1f2a-b538-4a74-bb56-79a13a821de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:34.997818 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0df4a-a925-4436-9601-a4d4d2eabea6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:34.999564 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0df4a-a925-4436-9601-a4d4d2eabea6 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:35.005677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:35.011726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 923/1841] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:34 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:35.018764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebaa852-6faa-438b-a03d-e8b467946c36 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:35.018961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebaa852-6faa-438b-a03d-e8b467946c36 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:35.019242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebaa852-6faa-438b-a03d-e8b467946c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:35.022604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 919/1842] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:35.027270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d1f2a-b538-4a74-bb56-79a13a821de4 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:35.043263 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d1f2a-b538-4a74-bb56-79a13a821de4 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:35.046907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9213926-5c4b-4f08-b5cc-b4bc21b0178d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:35.047544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9213926-5c4b-4f08-b5cc-b4bc21b0178d admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:35.047820 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9213926-5c4b-4f08-b5cc-b4bc21b0178d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:35.048114 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9213926-5c4b-4f08-b5cc-b4bc21b0178d admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:35.048291 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-3ebaa852-6faa-438b-a03d-e8b467946c36 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:35.048881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9213926-5c4b-4f08-b5cc-b4bc21b0178d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:45:35.049109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9213926-5c4b-4f08-b5cc-b4bc21b0178d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:35.049644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-3ebaa852-6faa-438b-a03d-e8b467946c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:35.050803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9213926-5c4b-4f08-b5cc-b4bc21b0178d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:35.058209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 924/1843] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:35.063116 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebaa852-6faa-438b-a03d-e8b467946c36 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:35.065562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-3ebaa852-6faa-438b-a03d-e8b467946c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:35.070025 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 920/1844] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:45:35.070025 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e69110-e4ff-461a-9d8d-5cce31d2ae57 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:35.070025 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e69110-e4ff-461a-9d8d-5cce31d2ae57 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:35.070025 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e69110-e4ff-461a-9d8d-5cce31d2ae57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:35.089661 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebaa852-6faa-438b-a03d-e8b467946c36 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:35.094838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f1612-3a6c-4d5a-a055-82687aa75fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:35.095533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f1612-3a6c-4d5a-a055-82687aa75fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:35.095809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f1612-3a6c-4d5a-a055-82687aa75fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:35.096089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f1612-3a6c-4d5a-a055-82687aa75fca admin admin] PATH_INFO: `/v3/users/c8681b583db24403aa873f60c5f30a2f` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:35.097542 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f1612-3a6c-4d5a-a055-82687aa75fca admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8681b583db24403aa873f60c5f30a2f)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:35.099604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f1612-3a6c-4d5a-a055-82687aa75fca admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:35.101866 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a1e69110-e4ff-461a-9d8d-5cce31d2ae57 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:35.103127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a1e69110-e4ff-461a-9d8d-5cce31d2ae57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:35.111162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 921/1845] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:35.112011 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e69110-e4ff-461a-9d8d-5cce31d2ae57 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:35.122524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e69110-e4ff-461a-9d8d-5cce31d2ae57 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:35.127281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5eebd-108e-4a8b-aadf-aef5cd4ee898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:35.127904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5eebd-108e-4a8b-aadf-aef5cd4ee898 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:35.128179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5eebd-108e-4a8b-aadf-aef5cd4ee898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:35.128880 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5eebd-108e-4a8b-aadf-aef5cd4ee898 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:35.129536 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5eebd-108e-4a8b-aadf-aef5cd4ee898 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:35.131449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5eebd-108e-4a8b-aadf-aef5cd4ee898 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:35.167157 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 922/1846] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:35 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:35.168034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5eebd-108e-4a8b-aadf-aef5cd4ee898 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:35.183413 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-ca3f1612-3a6c-4d5a-a055-82687aa75fca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8681b583db24403aa873f60c5f30a2f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:35.195741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-ca3f1612-3a6c-4d5a-a055-82687aa75fca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8681b583db24403aa873f60c5f30a2f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:35.196305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-ca3f1612-3a6c-4d5a-a055-82687aa75fca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8681b583db24403aa873f60c5f30a2f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:35.197038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-ca3f1612-3a6c-4d5a-a055-82687aa75fca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8681b583db24403aa873f60c5f30a2f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:35.210707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-ca3f1612-3a6c-4d5a-a055-82687aa75fca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8681b583db24403aa873f60c5f30a2f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:35.258913 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 925/1847] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:35 2022] DELETE /identity/v3/users/c8681b583db24403aa873f60c5f30a2f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:35.259840 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f1612-3a6c-4d5a-a055-82687aa75fca admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:35.266320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ca3f1612-3a6c-4d5a-a055-82687aa75fca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:35.269344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5eebd-108e-4a8b-aadf-aef5cd4ee898 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:35.307883 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:35.351395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630014d-5575-4b53-9ad2-898752424d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:35.352389 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630014d-5575-4b53-9ad2-898752424d08 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:35.352671 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630014d-5575-4b53-9ad2-898752424d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:35.352947 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630014d-5575-4b53-9ad2-898752424d08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:35.354674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630014d-5575-4b53-9ad2-898752424d08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:45:35.355002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630014d-5575-4b53-9ad2-898752424d08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:35.357189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630014d-5575-4b53-9ad2-898752424d08 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:35.378095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 923/1848] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:45:35.382804 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98592656-25ef-4b1b-ba69-fc50e17e463f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:35.383319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98592656-25ef-4b1b-ba69-fc50e17e463f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:35.383789 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f1612-3a6c-4d5a-a055-82687aa75fca admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:35.383911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98592656-25ef-4b1b-ba69-fc50e17e463f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:35.389439 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe942538-6714-4ead-ba09-e6fb764be5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:35.390106 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe942538-6714-4ead-ba09-e6fb764be5d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:35.390434 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe942538-6714-4ead-ba09-e6fb764be5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:35.390647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe942538-6714-4ead-ba09-e6fb764be5d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:35.391801 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe942538-6714-4ead-ba09-e6fb764be5d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:35.393568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe942538-6714-4ead-ba09-e6fb764be5d8 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:35.416100 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-98592656-25ef-4b1b-ba69-fc50e17e463f None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:35.418053 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-98592656-25ef-4b1b-ba69-fc50e17e463f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:35.427408 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 926/1849] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:35 2022] POST /identity/v3/users => generated 287 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:35.429031 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe942538-6714-4ead-ba09-e6fb764be5d8 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:35.432924 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-fe942538-6714-4ead-ba09-e6fb764be5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:35.433921 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 924/1850] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:35.434992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98592656-25ef-4b1b-ba69-fc50e17e463f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:35.448984 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98592656-25ef-4b1b-ba69-fc50e17e463f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:35.464215 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ab650-6fbe-426c-8c04-0e15a14e135b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:35.465399 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe942538-6714-4ead-ba09-e6fb764be5d8 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:35.466688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ab650-6fbe-426c-8c04-0e15a14e135b admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:35.467220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ab650-6fbe-426c-8c04-0e15a14e135b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:35.467788 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ab650-6fbe-426c-8c04-0e15a14e135b admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:35.469529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ab650-6fbe-426c-8c04-0e15a14e135b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:35.472291 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ab650-6fbe-426c-8c04-0e15a14e135b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:35.473576 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81ea60-f0db-45e6-9d9c-ac253ed32316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:35.474319 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81ea60-f0db-45e6-9d9c-ac253ed32316 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:35.474695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81ea60-f0db-45e6-9d9c-ac253ed32316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:35.475209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81ea60-f0db-45e6-9d9c-ac253ed32316 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:35.476480 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81ea60-f0db-45e6-9d9c-ac253ed32316 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:35.478256 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81ea60-f0db-45e6-9d9c-ac253ed32316 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:35.487395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 925/1851] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:35.488463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ab650-6fbe-426c-8c04-0e15a14e135b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:35.490706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-054ab650-6fbe-426c-8c04-0e15a14e135b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:35.527621 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 927/1852] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:35 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:35.528659 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81ea60-f0db-45e6-9d9c-ac253ed32316 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:35.567812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ab650-6fbe-426c-8c04-0e15a14e135b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:35.585295 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:35.640947 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81ea60-f0db-45e6-9d9c-ac253ed32316 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:35.646991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fed532-19e9-4cad-a415-750d28fc08ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:35.647742 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fed532-19e9-4cad-a415-750d28fc08ee admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:35.648081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fed532-19e9-4cad-a415-750d28fc08ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:35.648418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fed532-19e9-4cad-a415-750d28fc08ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:35.649274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fed532-19e9-4cad-a415-750d28fc08ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:35.651538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fed532-19e9-4cad-a415-750d28fc08ee admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:35.661948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7af5b-5d0a-4bc7-a73e-68a057edd8ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:35.662684 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7af5b-5d0a-4bc7-a73e-68a057edd8ce service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:35.663030 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7af5b-5d0a-4bc7-a73e-68a057edd8ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:35.663376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7af5b-5d0a-4bc7-a73e-68a057edd8ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:35.664702 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 928/1853] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:35.668592 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fed532-19e9-4cad-a415-750d28fc08ee admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:35.676590 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7af5b-5d0a-4bc7-a73e-68a057edd8ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:35.678709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7af5b-5d0a-4bc7-a73e-68a057edd8ce service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:35.680592 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fed532-19e9-4cad-a415-750d28fc08ee admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:35.688460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3749c5-cc99-429b-8ed4-30433a26891b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:35.689251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3749c5-cc99-429b-8ed4-30433a26891b admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:35.689906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3749c5-cc99-429b-8ed4-30433a26891b admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:35.689906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3749c5-cc99-429b-8ed4-30433a26891b admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:35.690882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3749c5-cc99-429b-8ed4-30433a26891b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:35.692643 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 926/1854] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:35.693567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3749c5-cc99-429b-8ed4-30433a26891b admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:35.693774 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7af5b-5d0a-4bc7-a73e-68a057edd8ce service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:35.704417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7af5b-5d0a-4bc7-a73e-68a057edd8ce service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:35.711336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d53dbb-7bfd-4661-a126-ff2f6bdf2fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:35.712690 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d53dbb-7bfd-4661-a126-ff2f6bdf2fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:35.712910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d53dbb-7bfd-4661-a126-ff2f6bdf2fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:35.713208 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d53dbb-7bfd-4661-a126-ff2f6bdf2fc1 admin admin] PATH_INFO: `/v3/projects/2718831f79ad4359a9dcd38185e42f20/users/9fcd56ecfc9e4adab5c7c63c0c1375c0/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:35.724167 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 929/1855] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:35 2022] POST /identity/v3/users => generated 294 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:35.729744 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3749c5-cc99-429b-8ed4-30433a26891b admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:35.738123 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d53dbb-7bfd-4661-a126-ff2f6bdf2fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2718831f79ad4359a9dcd38185e42f20, user_id=9fcd56ecfc9e4adab5c7c63c0c1375c0, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:35.741123 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d53dbb-7bfd-4661-a126-ff2f6bdf2fc1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:35.743428 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3749c5-cc99-429b-8ed4-30433a26891b admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:35.749718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533903b-2d9d-4eb6-bc45-598eaf326ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:35.749999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533903b-2d9d-4eb6-bc45-598eaf326ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:35.750269 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533903b-2d9d-4eb6-bc45-598eaf326ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:35.750784 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533903b-2d9d-4eb6-bc45-598eaf326ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:35.752356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533903b-2d9d-4eb6-bc45-598eaf326ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:35.754787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533903b-2d9d-4eb6-bc45-598eaf326ee2 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:35.771518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 927/1856] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:35 2022] PUT /identity/v3/projects/2718831f79ad4359a9dcd38185e42f20/users/9fcd56ecfc9e4adab5c7c63c0c1375c0/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:35.776798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 930/1857] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:45:35.778641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d53dbb-7bfd-4661-a126-ff2f6bdf2fc1 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:35.782906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e3999-10eb-4759-bb81-158309e79ebe None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:35.783341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e3999-10eb-4759-bb81-158309e79ebe None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:35.784114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e3999-10eb-4759-bb81-158309e79ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:35.869761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-326e3999-10eb-4759-bb81-158309e79ebe None None] MFA Rules not processed for user `9fcd56ecfc9e4adab5c7c63c0c1375c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:35.871462 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-326e3999-10eb-4759-bb81-158309e79ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:35.891077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:35.931448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d53dbb-7bfd-4661-a126-ff2f6bdf2fc1 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:35.938178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69072410-b2ea-41d8-b0cc-19726d528de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:35.939265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69072410-b2ea-41d8-b0cc-19726d528de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:35.939883 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69072410-b2ea-41d8-b0cc-19726d528de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:35.940625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69072410-b2ea-41d8-b0cc-19726d528de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:35.941640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69072410-b2ea-41d8-b0cc-19726d528de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:35.943679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69072410-b2ea-41d8-b0cc-19726d528de1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:35.956894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 928/1858] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:35.963163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69072410-b2ea-41d8-b0cc-19726d528de1 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:35.990715 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69072410-b2ea-41d8-b0cc-19726d528de1 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:35.998002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf360d-6d9f-44c1-8c6b-a8405f5a4e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:35.999189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf360d-6d9f-44c1-8c6b-a8405f5a4e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:36.000033 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf360d-6d9f-44c1-8c6b-a8405f5a4e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:36.000702 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf360d-6d9f-44c1-8c6b-a8405f5a4e32 admin admin] PATH_INFO: `/v3/projects/be54fd629aba4d4dad3ec17c658bf900/users/10b3dffba4f64c84b243f3cf21285ccb/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:36.043029 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 931/1859] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:45:36.047255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf360d-6d9f-44c1-8c6b-a8405f5a4e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be54fd629aba4d4dad3ec17c658bf900, user_id=10b3dffba4f64c84b243f3cf21285ccb, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:36.052332 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf360d-6d9f-44c1-8c6b-a8405f5a4e32 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:36.054451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e3999-10eb-4759-bb81-158309e79ebe None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:36.056858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-326e3999-10eb-4759-bb81-158309e79ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:36.093807 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 929/1860] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:35 2022] PUT /identity/v3/projects/be54fd629aba4d4dad3ec17c658bf900/users/10b3dffba4f64c84b243f3cf21285ccb/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:36.123781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508a2da-a644-49e7-9edc-fa4c172ca8d6 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:36.124165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508a2da-a644-49e7-9edc-fa4c172ca8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:36.124632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508a2da-a644-49e7-9edc-fa4c172ca8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:36.170067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e3999-10eb-4759-bb81-158309e79ebe None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:36.177511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-1508a2da-a644-49e7-9edc-fa4c172ca8d6 None None] MFA Rules not processed for user `10b3dffba4f64c84b243f3cf21285ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:36.179876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1508a2da-a644-49e7-9edc-fa4c172ca8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:36.269108 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:36.275603 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c07b71-5be0-47a9-9137-6cb1a25ea560 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:36.276249 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c07b71-5be0-47a9-9137-6cb1a25ea560 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:36.276525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c07b71-5be0-47a9-9137-6cb1a25ea560 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:36.276795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c07b71-5be0-47a9-9137-6cb1a25ea560 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:36.319422 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 930/1861] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:45:36.319422 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508a2da-a644-49e7-9edc-fa4c172ca8d6 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:36.319422 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1508a2da-a644-49e7-9edc-fa4c172ca8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:36.368236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c07b71-5be0-47a9-9137-6cb1a25ea560 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:36.369756 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c07b71-5be0-47a9-9137-6cb1a25ea560 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:36.381764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 932/1862] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:36.381764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c07b71-5be0-47a9-9137-6cb1a25ea560 service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:36.442127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508a2da-a644-49e7-9edc-fa4c172ca8d6 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:36.451929 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4f92f-70a0-4968-881d-051ab68164c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:36.452603 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4f92f-70a0-4968-881d-051ab68164c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:36.452979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4f92f-70a0-4968-881d-051ab68164c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:36.453485 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e4f92f-70a0-4968-881d-051ab68164c8 admin admin] PATH_INFO: `/v3/projects/edced8c05f4a4e8cb85b44c49505a829` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:36.455631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4f92f-70a0-4968-881d-051ab68164c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edced8c05f4a4e8cb85b44c49505a829)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:36.459655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e4f92f-70a0-4968-881d-051ab68164c8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:36.506708 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c07b71-5be0-47a9-9137-6cb1a25ea560 service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:36.528792 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:36.531279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-15e4f92f-70a0-4968-881d-051ab68164c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edced8c05f4a4e8cb85b44c49505a829', 'request_id': 'req-15e4f92f-70a0-4968-881d-051ab68164c8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:36.544482 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-15e4f92f-70a0-4968-881d-051ab68164c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edced8c05f4a4e8cb85b44c49505a829', 'request_id': 'req-15e4f92f-70a0-4968-881d-051ab68164c8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:36.557660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-15e4f92f-70a0-4968-881d-051ab68164c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edced8c05f4a4e8cb85b44c49505a829', 'request_id': 'req-15e4f92f-70a0-4968-881d-051ab68164c8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:36.576865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-15e4f92f-70a0-4968-881d-051ab68164c8 admin admin] The token cache is being invalidate because project edced8c05f4a4e8cb85b44c49505a829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:36.577186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-15e4f92f-70a0-4968-881d-051ab68164c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:36.578616 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-15e4f92f-70a0-4968-881d-051ab68164c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:36.583042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 931/1863] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:36 2022] DELETE /identity/v3/projects/edced8c05f4a4e8cb85b44c49505a829 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:36.583042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4f92f-70a0-4968-881d-051ab68164c8 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:36.583042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-15e4f92f-70a0-4968-881d-051ab68164c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:36.633406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe4656-fd37-4a83-87fc-02b89de9b553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:36.634560 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe4656-fd37-4a83-87fc-02b89de9b553 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:36.635174 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe4656-fd37-4a83-87fc-02b89de9b553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:36.636170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe4656-fd37-4a83-87fc-02b89de9b553 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:36.637480 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe4656-fd37-4a83-87fc-02b89de9b553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:36.640973 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe4656-fd37-4a83-87fc-02b89de9b553 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:36.669436 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e4f92f-70a0-4968-881d-051ab68164c8 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:36.683775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 933/1864] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:36 2022] POST /identity/v3/projects => generated 387 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:36.684555 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebbb74-d22f-458e-97a8-9ba1c4c4ed8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:36.685365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe4656-fd37-4a83-87fc-02b89de9b553 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:36.685890 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebbb74-d22f-458e-97a8-9ba1c4c4ed8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:36.686181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebbb74-d22f-458e-97a8-9ba1c4c4ed8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:36.686469 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebbb74-d22f-458e-97a8-9ba1c4c4ed8e admin admin] PATH_INFO: `/v3/users/9fcd56ecfc9e4adab5c7c63c0c1375c0` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:36.687982 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-b7fe4656-fd37-4a83-87fc-02b89de9b553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:36.689980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebbb74-d22f-458e-97a8-9ba1c4c4ed8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fcd56ecfc9e4adab5c7c63c0c1375c0)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:36.692384 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebbb74-d22f-458e-97a8-9ba1c4c4ed8e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:36.758181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:36.784278 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-99ebbb74-d22f-458e-97a8-9ba1c4c4ed8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fcd56ecfc9e4adab5c7c63c0c1375c0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:36.794394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-99ebbb74-d22f-458e-97a8-9ba1c4c4ed8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fcd56ecfc9e4adab5c7c63c0c1375c0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:36.794820 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-99ebbb74-d22f-458e-97a8-9ba1c4c4ed8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fcd56ecfc9e4adab5c7c63c0c1375c0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:36.795753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-99ebbb74-d22f-458e-97a8-9ba1c4c4ed8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fcd56ecfc9e4adab5c7c63c0c1375c0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:36.811113 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-99ebbb74-d22f-458e-97a8-9ba1c4c4ed8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fcd56ecfc9e4adab5c7c63c0c1375c0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:36.827407 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 932/1865] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:36 2022] DELETE /identity/v3/users/9fcd56ecfc9e4adab5c7c63c0c1375c0 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:36.828324 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebbb74-d22f-458e-97a8-9ba1c4c4ed8e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:36.830282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe4656-fd37-4a83-87fc-02b89de9b553 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:36.830760 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-99ebbb74-d22f-458e-97a8-9ba1c4c4ed8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:36.926377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1a983-74ed-4e8f-8cbc-436517cb67ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:36.927600 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1a983-74ed-4e8f-8cbc-436517cb67ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:36.927909 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1a983-74ed-4e8f-8cbc-436517cb67ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:36.928208 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1a983-74ed-4e8f-8cbc-436517cb67ad admin admin] PATH_INFO: `/v3/users/990740b9f848425cb1dac419b6a4e2a3` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:36.929838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1a983-74ed-4e8f-8cbc-436517cb67ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=990740b9f848425cb1dac419b6a4e2a3)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:36.932277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1a983-74ed-4e8f-8cbc-436517cb67ad admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:36.951845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:36.960610 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebbb74-d22f-458e-97a8-9ba1c4c4ed8e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:36.973464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f894f05-029a-43bc-8079-96ef9a63cad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:36.974597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f894f05-029a-43bc-8079-96ef9a63cad8 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:36.975078 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f894f05-029a-43bc-8079-96ef9a63cad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:36.975608 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f894f05-029a-43bc-8079-96ef9a63cad8 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:36.977307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f894f05-029a-43bc-8079-96ef9a63cad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:36.981648 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f894f05-029a-43bc-8079-96ef9a63cad8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:37.013437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f8d1a983-74ed-4e8f-8cbc-436517cb67ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '990740b9f848425cb1dac419b6a4e2a3'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:37.029774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f8d1a983-74ed-4e8f-8cbc-436517cb67ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '990740b9f848425cb1dac419b6a4e2a3'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:37.030472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f8d1a983-74ed-4e8f-8cbc-436517cb67ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '990740b9f848425cb1dac419b6a4e2a3'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:37.031293 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f8d1a983-74ed-4e8f-8cbc-436517cb67ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '990740b9f848425cb1dac419b6a4e2a3'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:37.033528 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 933/1866] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:36 2022] POST /identity/v3/users => generated 300 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:37.036482 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2cb56-8209-480b-9823-1649714829b3 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:37.037168 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2cb56-8209-480b-9823-1649714829b3 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:37.037609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2cb56-8209-480b-9823-1649714829b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:37.049708 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f8d1a983-74ed-4e8f-8cbc-436517cb67ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '990740b9f848425cb1dac419b6a4e2a3'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:37.083124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-20f2cb56-8209-480b-9823-1649714829b3 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:37.085495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-20f2cb56-8209-480b-9823-1649714829b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:37.096471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 934/1867] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:36 2022] DELETE /identity/v3/users/990740b9f848425cb1dac419b6a4e2a3 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:37.097759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1a983-74ed-4e8f-8cbc-436517cb67ad admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:37.101911 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f8d1a983-74ed-4e8f-8cbc-436517cb67ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:37.110723 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 934/1868] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:37.123467 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2cb56-8209-480b-9823-1649714829b3 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:37.126343 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-20f2cb56-8209-480b-9823-1649714829b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:37.275034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2cb56-8209-480b-9823-1649714829b3 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:37.297238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1a983-74ed-4e8f-8cbc-436517cb67ad admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:37.311960 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b5ad8-48c2-4d14-b2a0-4bde0d269874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:37.312885 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b5ad8-48c2-4d14-b2a0-4bde0d269874 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:37.313553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b5ad8-48c2-4d14-b2a0-4bde0d269874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:37.314314 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b5ad8-48c2-4d14-b2a0-4bde0d269874 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:37.315457 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b5ad8-48c2-4d14-b2a0-4bde0d269874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:37.318937 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b5ad8-48c2-4d14-b2a0-4bde0d269874 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:37.321022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:37.344084 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 935/1869] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:37.349081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b5ad8-48c2-4d14-b2a0-4bde0d269874 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:37.381032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b5ad8-48c2-4d14-b2a0-4bde0d269874 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:37.395931 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623c35c-faad-4a65-8202-4d843ecbbe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:37.397407 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623c35c-faad-4a65-8202-4d843ecbbe2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:37.397532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623c35c-faad-4a65-8202-4d843ecbbe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:37.397939 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623c35c-faad-4a65-8202-4d843ecbbe2d admin admin] PATH_INFO: `/v3/projects/0ddce1bc9f3845e6b4362458d5f99ca0/users/c520cc46a6df46ae93571116b23f56e7/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:37.402185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ca100-fa2a-4f79-8a55-bcd44a1ae140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:37.403624 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ca100-fa2a-4f79-8a55-bcd44a1ae140 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:37.404066 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ca100-fa2a-4f79-8a55-bcd44a1ae140 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:37.404508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ca100-fa2a-4f79-8a55-bcd44a1ae140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:37.442996 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ca100-fa2a-4f79-8a55-bcd44a1ae140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:37.449177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623c35c-faad-4a65-8202-4d843ecbbe2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ddce1bc9f3845e6b4362458d5f99ca0, user_id=c520cc46a6df46ae93571116b23f56e7, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:37.453512 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623c35c-faad-4a65-8202-4d843ecbbe2d admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:37.454496 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ca100-fa2a-4f79-8a55-bcd44a1ae140 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:37.478055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 935/1870] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:37.490482 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 936/1871] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:37 2022] PUT /identity/v3/projects/0ddce1bc9f3845e6b4362458d5f99ca0/users/c520cc46a6df46ae93571116b23f56e7/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:45:37.496474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ca100-fa2a-4f79-8a55-bcd44a1ae140 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:37.523319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ca100-fa2a-4f79-8a55-bcd44a1ae140 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:37.640568 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa264c-5386-4af7-abc2-2a25093ca84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:37.641514 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa264c-5386-4af7-abc2-2a25093ca84d admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:37.641978 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa264c-5386-4af7-abc2-2a25093ca84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:37.642331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa264c-5386-4af7-abc2-2a25093ca84d admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:37.643183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa264c-5386-4af7-abc2-2a25093ca84d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:37.647702 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa264c-5386-4af7-abc2-2a25093ca84d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:37.667183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 936/1872] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:37.667770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623c35c-faad-4a65-8202-4d843ecbbe2d admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:37.668710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa264c-5386-4af7-abc2-2a25093ca84d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:37.670912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4623c35c-faad-4a65-8202-4d843ecbbe2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:37.687116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa264c-5386-4af7-abc2-2a25093ca84d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:37.697531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae308b1-9e89-48cd-a8bf-c147ecbb36dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:37.698580 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae308b1-9e89-48cd-a8bf-c147ecbb36dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:37.699041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae308b1-9e89-48cd-a8bf-c147ecbb36dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:37.699504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae308b1-9e89-48cd-a8bf-c147ecbb36dc admin admin] PATH_INFO: `/v3/projects/0ddce1bc9f3845e6b4362458d5f99ca0/users/c520cc46a6df46ae93571116b23f56e7/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:37.702844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae308b1-9e89-48cd-a8bf-c147ecbb36dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ddce1bc9f3845e6b4362458d5f99ca0, user_id=c520cc46a6df46ae93571116b23f56e7, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:37.707183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae308b1-9e89-48cd-a8bf-c147ecbb36dc admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:37.725285 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623c35c-faad-4a65-8202-4d843ecbbe2d admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:37.743264 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 937/1873] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:37 2022] PUT /identity/v3/projects/0ddce1bc9f3845e6b4362458d5f99ca0/users/c520cc46a6df46ae93571116b23f56e7/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:37.749970 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f79fdf-1052-46c8-b73e-18e7252d708d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:37.750431 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f79fdf-1052-46c8-b73e-18e7252d708d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:37.751030 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f79fdf-1052-46c8-b73e-18e7252d708d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:37.864163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b8f79fdf-1052-46c8-b73e-18e7252d708d None None] MFA Rules not processed for user `c520cc46a6df46ae93571116b23f56e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:37.865550 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b8f79fdf-1052-46c8-b73e-18e7252d708d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:37.892554 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:37.906699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c054da9-4c30-47fe-9142-9bb14a6a5f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:37.907490 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c054da9-4c30-47fe-9142-9bb14a6a5f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:37.908338 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c054da9-4c30-47fe-9142-9bb14a6a5f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:37.908714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c054da9-4c30-47fe-9142-9bb14a6a5f25 admin admin] PATH_INFO: `/v3/projects/ea7c92c066e548fe9444295d11dc22fc` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:37.910947 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c054da9-4c30-47fe-9142-9bb14a6a5f25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea7c92c066e548fe9444295d11dc22fc)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:37.915667 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c054da9-4c30-47fe-9142-9bb14a6a5f25 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:38.010000 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1c054da9-4c30-47fe-9142-9bb14a6a5f25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea7c92c066e548fe9444295d11dc22fc', 'request_id': 'req-1c054da9-4c30-47fe-9142-9bb14a6a5f25'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:38.026914 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 938/1874] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:38.029879 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1c054da9-4c30-47fe-9142-9bb14a6a5f25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea7c92c066e548fe9444295d11dc22fc', 'request_id': 'req-1c054da9-4c30-47fe-9142-9bb14a6a5f25'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:38.047354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f79fdf-1052-46c8-b73e-18e7252d708d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:38.051441 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b8f79fdf-1052-46c8-b73e-18e7252d708d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:38.055297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1c054da9-4c30-47fe-9142-9bb14a6a5f25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea7c92c066e548fe9444295d11dc22fc', 'request_id': 'req-1c054da9-4c30-47fe-9142-9bb14a6a5f25'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:38.076177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1c054da9-4c30-47fe-9142-9bb14a6a5f25 admin admin] The token cache is being invalidate because project ea7c92c066e548fe9444295d11dc22fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:38.076617 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1c054da9-4c30-47fe-9142-9bb14a6a5f25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:38.077748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1c054da9-4c30-47fe-9142-9bb14a6a5f25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:38.080292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 937/1875] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:37 2022] DELETE /identity/v3/projects/ea7c92c066e548fe9444295d11dc22fc => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:38.088274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c0901-29a6-4ce0-96db-ed0074e79f9c None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:38.088570 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c0901-29a6-4ce0-96db-ed0074e79f9c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:38.088760 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c0901-29a6-4ce0-96db-ed0074e79f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:38.134717 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-f14c0901-29a6-4ce0-96db-ed0074e79f9c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:38.136822 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f14c0901-29a6-4ce0-96db-ed0074e79f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:38.172708 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f79fdf-1052-46c8-b73e-18e7252d708d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:38.260814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 938/1876] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:45:38.262226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c0901-29a6-4ce0-96db-ed0074e79f9c None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:38.263861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f14c0901-29a6-4ce0-96db-ed0074e79f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:38.271761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:38.278138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c0901-29a6-4ce0-96db-ed0074e79f9c None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:38.292029 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65267335-8d17-48c7-834b-22c5544a30af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:38.292540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65267335-8d17-48c7-834b-22c5544a30af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:38.293263 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65267335-8d17-48c7-834b-22c5544a30af admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:38.293919 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65267335-8d17-48c7-834b-22c5544a30af admin admin] PATH_INFO: `/v3/projects/2718831f79ad4359a9dcd38185e42f20` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:38.294576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c318c8-66ef-4747-9f04-51ee8a9c87c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:38.295214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c318c8-66ef-4747-9f04-51ee8a9c87c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:38.297065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65267335-8d17-48c7-834b-22c5544a30af admin admin] RBAC: Authorizing `identity:delete_project(project_id=2718831f79ad4359a9dcd38185e42f20)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:38.297243 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c318c8-66ef-4747-9f04-51ee8a9c87c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:38.297243 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c318c8-66ef-4747-9f04-51ee8a9c87c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:38.298471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-09c318c8-66ef-4747-9f04-51ee8a9c87c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:38.299939 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65267335-8d17-48c7-834b-22c5544a30af admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:38.435765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-65267335-8d17-48c7-834b-22c5544a30af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2718831f79ad4359a9dcd38185e42f20', 'request_id': 'req-65267335-8d17-48c7-834b-22c5544a30af'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:38.462589 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-65267335-8d17-48c7-834b-22c5544a30af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2718831f79ad4359a9dcd38185e42f20', 'request_id': 'req-65267335-8d17-48c7-834b-22c5544a30af'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:38.490357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-65267335-8d17-48c7-834b-22c5544a30af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2718831f79ad4359a9dcd38185e42f20', 'request_id': 'req-65267335-8d17-48c7-834b-22c5544a30af'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:38.513363 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c318c8-66ef-4747-9f04-51ee8a9c87c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:38.518068 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c318c8-66ef-4747-9f04-51ee8a9c87c0 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:38.522133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-65267335-8d17-48c7-834b-22c5544a30af admin admin] The token cache is being invalidate because project 2718831f79ad4359a9dcd38185e42f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:38.522767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-65267335-8d17-48c7-834b-22c5544a30af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:38.523989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-65267335-8d17-48c7-834b-22c5544a30af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:38.526960 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 939/1877] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:38 2022] DELETE /identity/v3/projects/2718831f79ad4359a9dcd38185e42f20 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:38.528913 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac996a3-d5bb-4fd5-a3eb-def312f65f20 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:38.529578 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac996a3-d5bb-4fd5-a3eb-def312f65f20 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:38.530279 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac996a3-d5bb-4fd5-a3eb-def312f65f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:38.542436 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 939/1878] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:38.544912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d312b-6323-4e23-bb8b-6f1988c894af None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:38.545400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d312b-6323-4e23-bb8b-6f1988c894af None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:38.545846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d312b-6323-4e23-bb8b-6f1988c894af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:38.585997 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-6ac996a3-d5bb-4fd5-a3eb-def312f65f20 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:38.587726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6ac996a3-d5bb-4fd5-a3eb-def312f65f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:38.589902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c41d312b-6323-4e23-bb8b-6f1988c894af None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:38.591095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c41d312b-6323-4e23-bb8b-6f1988c894af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:38.600450 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:38.740687 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 940/1879] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:38.747410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac996a3-d5bb-4fd5-a3eb-def312f65f20 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:38.760399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 940/1880] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:38.772018 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e73f0-b4f8-4190-9f45-6032a08a4713 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:38.772462 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e73f0-b4f8-4190-9f45-6032a08a4713 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:38.772897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e73f0-b4f8-4190-9f45-6032a08a4713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:38.787033 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac996a3-d5bb-4fd5-a3eb-def312f65f20 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:38.794261 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67334998-c88b-492c-9c8b-c5f3fc04e6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:38.795278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67334998-c88b-492c-9c8b-c5f3fc04e6cd admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:38.795749 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67334998-c88b-492c-9c8b-c5f3fc04e6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:38.796435 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67334998-c88b-492c-9c8b-c5f3fc04e6cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:38.798048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67334998-c88b-492c-9c8b-c5f3fc04e6cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:45:38.798600 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67334998-c88b-492c-9c8b-c5f3fc04e6cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:38.801996 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67334998-c88b-492c-9c8b-c5f3fc04e6cd admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:38.828723 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-935e73f0-b4f8-4190-9f45-6032a08a4713 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:38.837118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-935e73f0-b4f8-4190-9f45-6032a08a4713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:38.842397 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 941/1881] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:45:38.844356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67334998-c88b-492c-9c8b-c5f3fc04e6cd admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:38.848022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-67334998-c88b-492c-9c8b-c5f3fc04e6cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:38.858045 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 941/1882] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:45:38.859772 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e73f0-b4f8-4190-9f45-6032a08a4713 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:38.874285 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e73f0-b4f8-4190-9f45-6032a08a4713 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:38.875390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67334998-c88b-492c-9c8b-c5f3fc04e6cd admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:38.883432 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bea326-d235-46d7-aa2e-120f22089a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:38.884958 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bea326-d235-46d7-aa2e-120f22089a5e admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:38.885179 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278471d7-8c6b-4de7-a096-7d070d19bfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:38.885937 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bea326-d235-46d7-aa2e-120f22089a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:38.886442 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bea326-d235-46d7-aa2e-120f22089a5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:38.887040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278471d7-8c6b-4de7-a096-7d070d19bfbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:38.887529 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278471d7-8c6b-4de7-a096-7d070d19bfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:38.887684 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bea326-d235-46d7-aa2e-120f22089a5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:38.888009 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278471d7-8c6b-4de7-a096-7d070d19bfbe admin admin] PATH_INFO: `/v3/users/10b3dffba4f64c84b243f3cf21285ccb` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:38.891639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278471d7-8c6b-4de7-a096-7d070d19bfbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=10b3dffba4f64c84b243f3cf21285ccb)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:38.891851 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bea326-d235-46d7-aa2e-120f22089a5e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:38.895739 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278471d7-8c6b-4de7-a096-7d070d19bfbe admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:38.965555 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 942/1883] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:38 2022] POST /identity/v3/projects => generated 399 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:38.967219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bea326-d235-46d7-aa2e-120f22089a5e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:38.999899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-278471d7-8c6b-4de7-a096-7d070d19bfbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10b3dffba4f64c84b243f3cf21285ccb'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:39.012263 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-278471d7-8c6b-4de7-a096-7d070d19bfbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10b3dffba4f64c84b243f3cf21285ccb'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:39.013734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-278471d7-8c6b-4de7-a096-7d070d19bfbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10b3dffba4f64c84b243f3cf21285ccb'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:39.015024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-278471d7-8c6b-4de7-a096-7d070d19bfbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10b3dffba4f64c84b243f3cf21285ccb'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:39.027887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-278471d7-8c6b-4de7-a096-7d070d19bfbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10b3dffba4f64c84b243f3cf21285ccb'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:39.034387 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:39.078057 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 942/1884] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:38 2022] DELETE /identity/v3/users/10b3dffba4f64c84b243f3cf21285ccb => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:39.081361 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278471d7-8c6b-4de7-a096-7d070d19bfbe admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:39.084142 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-278471d7-8c6b-4de7-a096-7d070d19bfbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:39.187831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bea326-d235-46d7-aa2e-120f22089a5e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:39.392370 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278471d7-8c6b-4de7-a096-7d070d19bfbe admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:39.401406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38063b2-7f29-4f85-8838-5e1c8204ace2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:39.403811 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38063b2-7f29-4f85-8838-5e1c8204ace2 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:39.404409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38063b2-7f29-4f85-8838-5e1c8204ace2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:39.405006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38063b2-7f29-4f85-8838-5e1c8204ace2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:39.405322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd0afd-e0cf-477a-a4a0-266b6c739c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:39.406671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd0afd-e0cf-477a-a4a0-266b6c739c8c admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:39.406855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38063b2-7f29-4f85-8838-5e1c8204ace2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:45:39.407546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd0afd-e0cf-477a-a4a0-266b6c739c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:39.408111 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38063b2-7f29-4f85-8838-5e1c8204ace2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:39.408416 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd0afd-e0cf-477a-a4a0-266b6c739c8c admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:39.410076 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd0afd-e0cf-477a-a4a0-266b6c739c8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:39.413913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38063b2-7f29-4f85-8838-5e1c8204ace2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:39.415876 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd0afd-e0cf-477a-a4a0-266b6c739c8c admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:39.451695 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 943/1885] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 481 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:45:39.456204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b1b86-0bf1-47c1-b25b-bf299c1c9c59 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:39.457175 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b1b86-0bf1-47c1-b25b-bf299c1c9c59 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:39.457865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b1b86-0bf1-47c1-b25b-bf299c1c9c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:39.473948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 943/1886] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:39 2022] POST /identity/v3/users => generated 300 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:39.473948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd0afd-e0cf-477a-a4a0-266b6c739c8c admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:39.476444 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c8dd0afd-e0cf-477a-a4a0-266b6c739c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:39.505229 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd0afd-e0cf-477a-a4a0-266b6c739c8c admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:39.505715 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-1a8b1b86-0bf1-47c1-b25b-bf299c1c9c59 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:39.509153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1a8b1b86-0bf1-47c1-b25b-bf299c1c9c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:39.517867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabacbee-b9a8-4377-a585-c0f39a34835a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:39.519359 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabacbee-b9a8-4377-a585-c0f39a34835a admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:39.520421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabacbee-b9a8-4377-a585-c0f39a34835a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:39.521281 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabacbee-b9a8-4377-a585-c0f39a34835a admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:39.522890 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabacbee-b9a8-4377-a585-c0f39a34835a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:39.525632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 944/1887] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:39.527719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabacbee-b9a8-4377-a585-c0f39a34835a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:39.528292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b1b86-0bf1-47c1-b25b-bf299c1c9c59 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:39.548572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b1b86-0bf1-47c1-b25b-bf299c1c9c59 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:39.558690 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59cfcf-3fe3-467b-a181-5691f7215199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:39.560113 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59cfcf-3fe3-467b-a181-5691f7215199 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:39.560755 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59cfcf-3fe3-467b-a181-5691f7215199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:39.561730 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59cfcf-3fe3-467b-a181-5691f7215199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:39.563696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59cfcf-3fe3-467b-a181-5691f7215199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:39.567739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59cfcf-3fe3-467b-a181-5691f7215199 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:39.599137 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 945/1888] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:39.601306 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59cfcf-3fe3-467b-a181-5691f7215199 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:39.605183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6f59cfcf-3fe3-467b-a181-5691f7215199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:39.616331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 944/1889] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:39 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:39.619688 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabacbee-b9a8-4377-a585-c0f39a34835a admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:39.633938 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:39.741884 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabacbee-b9a8-4377-a585-c0f39a34835a admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:39.747571 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e8c80-35f8-487e-a665-dca2f8c0f728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:39.748238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e8c80-35f8-487e-a665-dca2f8c0f728 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:39.748540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e8c80-35f8-487e-a665-dca2f8c0f728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:39.748820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e8c80-35f8-487e-a665-dca2f8c0f728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:39.749623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e8c80-35f8-487e-a665-dca2f8c0f728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:39.751422 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e8c80-35f8-487e-a665-dca2f8c0f728 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:39.771768 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59cfcf-3fe3-467b-a181-5691f7215199 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:39.806023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 945/1890] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:39.809844 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e8c80-35f8-487e-a665-dca2f8c0f728 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:39.845449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44b559-2d67-405d-8907-5b622dacfd33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:39.846358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44b559-2d67-405d-8907-5b622dacfd33 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:39.846645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44b559-2d67-405d-8907-5b622dacfd33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:39.846917 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44b559-2d67-405d-8907-5b622dacfd33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:39.851940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e8c80-35f8-487e-a665-dca2f8c0f728 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:39.863188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a86cf3-e88e-4759-b348-e295ce1ce85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:39.865016 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a86cf3-e88e-4759-b348-e295ce1ce85e admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:39.865796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a86cf3-e88e-4759-b348-e295ce1ce85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:39.866830 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a86cf3-e88e-4759-b348-e295ce1ce85e admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:39.868561 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a86cf3-e88e-4759-b348-e295ce1ce85e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:39.871312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44b559-2d67-405d-8907-5b622dacfd33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:39.872969 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44b559-2d67-405d-8907-5b622dacfd33 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:39.874456 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a86cf3-e88e-4759-b348-e295ce1ce85e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:39.902101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 946/1891] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:39.902767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44b559-2d67-405d-8907-5b622dacfd33 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:39.932552 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44b559-2d67-405d-8907-5b622dacfd33 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:39.942719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67617298-32f1-40aa-8a63-6e0d532e1d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:39.943421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67617298-32f1-40aa-8a63-6e0d532e1d23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:39.943700 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67617298-32f1-40aa-8a63-6e0d532e1d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:39.944407 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67617298-32f1-40aa-8a63-6e0d532e1d23 admin admin] PATH_INFO: `/v3/projects/e9938011d6254805be9c8b15d02be673/users/e45671b12f6f4284adfa0a3d1777f1e3/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:39.945064 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 946/1892] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:39 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:39.951845 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a86cf3-e88e-4759-b348-e295ce1ce85e admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:39.967311 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67617298-32f1-40aa-8a63-6e0d532e1d23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9938011d6254805be9c8b15d02be673, user_id=e45671b12f6f4284adfa0a3d1777f1e3, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:39.970529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67617298-32f1-40aa-8a63-6e0d532e1d23 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:39.973472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a86cf3-e88e-4759-b348-e295ce1ce85e admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:39.993821 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee9def-02e4-4a61-be37-e0c29fc521cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:39.994413 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 947/1893] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:39 2022] PUT /identity/v3/projects/e9938011d6254805be9c8b15d02be673/users/e45671b12f6f4284adfa0a3d1777f1e3/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:40.007140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee9def-02e4-4a61-be37-e0c29fc521cf admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:40.007723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee9def-02e4-4a61-be37-e0c29fc521cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:40.008180 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee9def-02e4-4a61-be37-e0c29fc521cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:40.009267 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee9def-02e4-4a61-be37-e0c29fc521cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:40.012931 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee9def-02e4-4a61-be37-e0c29fc521cf admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:40.030536 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520e152-7059-4d93-b15d-1837bbb87383 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:40.030829 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520e152-7059-4d93-b15d-1837bbb87383 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:40.031093 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520e152-7059-4d93-b15d-1837bbb87383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:40.036429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 947/1894] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:45:40.053660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee9def-02e4-4a61-be37-e0c29fc521cf admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:40.057235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee9def-02e4-4a61-be37-e0c29fc521cf admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:40.084667 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:DELETE=2,keystone:SELECT=91,keystone:INSERT=7 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:40.122944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c520e152-7059-4d93-b15d-1837bbb87383 None None] MFA Rules not processed for user `e45671b12f6f4284adfa0a3d1777f1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:40.124794 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c520e152-7059-4d93-b15d-1837bbb87383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:40.164562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da313a-1d64-45e1-9cb7-aeb5d5e2cd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:40.166282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da313a-1d64-45e1-9cb7-aeb5d5e2cd1f admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:40.166979 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da313a-1d64-45e1-9cb7-aeb5d5e2cd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:40.167681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da313a-1d64-45e1-9cb7-aeb5d5e2cd1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:40.168809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da313a-1d64-45e1-9cb7-aeb5d5e2cd1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:40.172231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da313a-1d64-45e1-9cb7-aeb5d5e2cd1f admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:40.199553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 948/1895] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:40.205280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da313a-1d64-45e1-9cb7-aeb5d5e2cd1f admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:40.231814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da313a-1d64-45e1-9cb7-aeb5d5e2cd1f admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:40.242436 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160a817-8d8e-4f29-b259-83620b510bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:40.244146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160a817-8d8e-4f29-b259-83620b510bac admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:40.245831 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160a817-8d8e-4f29-b259-83620b510bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:40.246595 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160a817-8d8e-4f29-b259-83620b510bac admin admin] PATH_INFO: `/v3/projects/4f484d66ba0b47e5a9c04d6f2b1bf88c/users/bda11de729f0434aacc89f9d23c705ff/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:40.273780 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 948/1896] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:40 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:40.275739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520e152-7059-4d93-b15d-1837bbb87383 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:40.285494 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8160a817-8d8e-4f29-b259-83620b510bac admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f484d66ba0b47e5a9c04d6f2b1bf88c, user_id=bda11de729f0434aacc89f9d23c705ff, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:40.293234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8160a817-8d8e-4f29-b259-83620b510bac admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:40.299868 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520e152-7059-4d93-b15d-1837bbb87383 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:40.309994 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c23216-d0cc-4193-92ca-dfcae0ef00d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:40.310324 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c23216-d0cc-4193-92ca-dfcae0ef00d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:40.310868 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c23216-d0cc-4193-92ca-dfcae0ef00d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:40.311402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c23216-d0cc-4193-92ca-dfcae0ef00d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:40.312631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c23216-d0cc-4193-92ca-dfcae0ef00d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:40.325421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c23216-d0cc-4193-92ca-dfcae0ef00d0 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:40.334490 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 949/1897] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:40 2022] PUT /identity/v3/projects/4f484d66ba0b47e5a9c04d6f2b1bf88c/users/bda11de729f0434aacc89f9d23c705ff/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:40.354352 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f29222-0669-4dfa-9fa3-01ad4f5731b6 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:40.354706 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f29222-0669-4dfa-9fa3-01ad4f5731b6 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:40.355229 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f29222-0669-4dfa-9fa3-01ad4f5731b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:40.413174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 949/1898] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:40 2022] POST /identity/v3/projects => generated 401 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:40.419188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c23216-d0cc-4193-92ca-dfcae0ef00d0 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:40.434801 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:40.460800 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-88f29222-0669-4dfa-9fa3-01ad4f5731b6 None None] MFA Rules not processed for user `bda11de729f0434aacc89f9d23c705ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:40.463807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-88f29222-0669-4dfa-9fa3-01ad4f5731b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:40.587205 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c23216-d0cc-4193-92ca-dfcae0ef00d0 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:40.593082 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 950/1899] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:40.598610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f29222-0669-4dfa-9fa3-01ad4f5731b6 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:40.599048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189e394-f115-4d51-90a5-e8c767531090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:40.599799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189e394-f115-4d51-90a5-e8c767531090 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:40.600182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189e394-f115-4d51-90a5-e8c767531090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:40.600524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189e394-f115-4d51-90a5-e8c767531090 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:40.601323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189e394-f115-4d51-90a5-e8c767531090 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:40.606996 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189e394-f115-4d51-90a5-e8c767531090 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:40.635209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f29222-0669-4dfa-9fa3-01ad4f5731b6 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:40.645482 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 950/1900] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:40 2022] POST /identity/v3/users => generated 307 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:40.646956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288092d-fca7-41c8-a090-01aa2321f5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:40.647720 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288092d-fca7-41c8-a090-01aa2321f5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:40.649082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189e394-f115-4d51-90a5-e8c767531090 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:40.651701 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288092d-fca7-41c8-a090-01aa2321f5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:40.651701 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288092d-fca7-41c8-a090-01aa2321f5d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:40.653527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288092d-fca7-41c8-a090-01aa2321f5d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:40.668518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189e394-f115-4d51-90a5-e8c767531090 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:40.669758 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288092d-fca7-41c8-a090-01aa2321f5d9 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:40.676224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998519c-4e7c-4d1b-bd21-baea20f99ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:40.677590 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998519c-4e7c-4d1b-bd21-baea20f99ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:40.678116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998519c-4e7c-4d1b-bd21-baea20f99ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:40.678658 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998519c-4e7c-4d1b-bd21-baea20f99ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:40.679786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998519c-4e7c-4d1b-bd21-baea20f99ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:40.682028 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998519c-4e7c-4d1b-bd21-baea20f99ee6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:40.702369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 951/1901] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:40.706459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998519c-4e7c-4d1b-bd21-baea20f99ee6 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:40.723435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998519c-4e7c-4d1b-bd21-baea20f99ee6 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:40.730366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938bbc33-7ca5-4317-9bf7-55d12538c373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:40.731439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938bbc33-7ca5-4317-9bf7-55d12538c373 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:40.733326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938bbc33-7ca5-4317-9bf7-55d12538c373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:40.733326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938bbc33-7ca5-4317-9bf7-55d12538c373 admin admin] PATH_INFO: `/v3/projects/95149a85bf3749f0b8c53722656c3a27/users/b7fd6c37fc1f47af8012af87ca705a5a/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:40.760124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938bbc33-7ca5-4317-9bf7-55d12538c373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95149a85bf3749f0b8c53722656c3a27, user_id=b7fd6c37fc1f47af8012af87ca705a5a, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:40.764709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938bbc33-7ca5-4317-9bf7-55d12538c373 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:40.782113 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 951/1902] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:40 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:40.788072 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288092d-fca7-41c8-a090-01aa2321f5d9 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:40.826219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 952/1903] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:40 2022] PUT /identity/v3/projects/95149a85bf3749f0b8c53722656c3a27/users/b7fd6c37fc1f47af8012af87ca705a5a/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:40.830442 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938bbc33-7ca5-4317-9bf7-55d12538c373 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:40.915710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288092d-fca7-41c8-a090-01aa2321f5d9 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:40.927352 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:41.053070 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938bbc33-7ca5-4317-9bf7-55d12538c373 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:41.063966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a7a49-7e51-4b68-ab46-2e887ba297fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:41.065173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a7a49-7e51-4b68-ab46-2e887ba297fb admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:41.065785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a7a49-7e51-4b68-ab46-2e887ba297fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:41.066343 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a7a49-7e51-4b68-ab46-2e887ba297fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:41.067697 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a7a49-7e51-4b68-ab46-2e887ba297fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:41.070657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23784e47-1050-4857-88a4-ee9271b6e01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:41.070657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23784e47-1050-4857-88a4-ee9271b6e01b admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:41.070657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23784e47-1050-4857-88a4-ee9271b6e01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:41.071239 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23784e47-1050-4857-88a4-ee9271b6e01b admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:41.073014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23784e47-1050-4857-88a4-ee9271b6e01b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:41.075647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23784e47-1050-4857-88a4-ee9271b6e01b admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:41.082144 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a7a49-7e51-4b68-ab46-2e887ba297fb admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:41.115241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 952/1904] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:40 2022] POST /identity/v3/users => generated 291 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:45:41.121275 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23784e47-1050-4857-88a4-ee9271b6e01b admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:41.124025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-23784e47-1050-4857-88a4-ee9271b6e01b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:41.138753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 953/1905] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:45:41.140059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a7a49-7e51-4b68-ab46-2e887ba297fb admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:41.148752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23784e47-1050-4857-88a4-ee9271b6e01b admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:41.155502 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107b8da-119c-4c34-98c2-467de09e78f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:41.156701 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107b8da-119c-4c34-98c2-467de09e78f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:41.157196 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107b8da-119c-4c34-98c2-467de09e78f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:41.157995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107b8da-119c-4c34-98c2-467de09e78f5 admin admin] PATH_INFO: `/v3/projects/be54fd629aba4d4dad3ec17c658bf900` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:41.159643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107b8da-119c-4c34-98c2-467de09e78f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be54fd629aba4d4dad3ec17c658bf900)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:41.162659 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107b8da-119c-4c34-98c2-467de09e78f5 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:41.172723 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a7a49-7e51-4b68-ab46-2e887ba297fb admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:41.178347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce4120-92a1-498a-a0d1-0c26fb27291e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:41.178988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce4120-92a1-498a-a0d1-0c26fb27291e admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:41.179264 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce4120-92a1-498a-a0d1-0c26fb27291e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:41.179535 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce4120-92a1-498a-a0d1-0c26fb27291e admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:41.180993 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce4120-92a1-498a-a0d1-0c26fb27291e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:41.182819 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce4120-92a1-498a-a0d1-0c26fb27291e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:41.203453 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 954/1906] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:41.203453 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce4120-92a1-498a-a0d1-0c26fb27291e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:41.216778 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce4120-92a1-498a-a0d1-0c26fb27291e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:41.233438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56d8f2-0845-4c73-bdf1-848b7864d074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:41.233438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56d8f2-0845-4c73-bdf1-848b7864d074 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:41.233438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56d8f2-0845-4c73-bdf1-848b7864d074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:41.233438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56d8f2-0845-4c73-bdf1-848b7864d074 admin admin] PATH_INFO: `/v3/projects/95149a85bf3749f0b8c53722656c3a27/users/b7fd6c37fc1f47af8012af87ca705a5a/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:41.237434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56d8f2-0845-4c73-bdf1-848b7864d074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95149a85bf3749f0b8c53722656c3a27, user_id=b7fd6c37fc1f47af8012af87ca705a5a, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:41.243158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56d8f2-0845-4c73-bdf1-848b7864d074 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:41.250331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8107b8da-119c-4c34-98c2-467de09e78f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be54fd629aba4d4dad3ec17c658bf900', 'request_id': 'req-8107b8da-119c-4c34-98c2-467de09e78f5'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:41.262376 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8107b8da-119c-4c34-98c2-467de09e78f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be54fd629aba4d4dad3ec17c658bf900', 'request_id': 'req-8107b8da-119c-4c34-98c2-467de09e78f5'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:41.269968 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 955/1907] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:41 2022] PUT /identity/v3/projects/95149a85bf3749f0b8c53722656c3a27/users/b7fd6c37fc1f47af8012af87ca705a5a/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:41.271705 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56d8f2-0845-4c73-bdf1-848b7864d074 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:41.288753 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8107b8da-119c-4c34-98c2-467de09e78f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be54fd629aba4d4dad3ec17c658bf900', 'request_id': 'req-8107b8da-119c-4c34-98c2-467de09e78f5'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:41.305044 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8107b8da-119c-4c34-98c2-467de09e78f5 admin admin] The token cache is being invalidate because project be54fd629aba4d4dad3ec17c658bf900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:41.305327 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8107b8da-119c-4c34-98c2-467de09e78f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:41.305987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8107b8da-119c-4c34-98c2-467de09e78f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:41.310324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 953/1908] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:41 2022] DELETE /identity/v3/projects/be54fd629aba4d4dad3ec17c658bf900 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:41.310894 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31bc509-040d-49cc-82c4-d97e64d27076 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:41.311172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31bc509-040d-49cc-82c4-d97e64d27076 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:41.311607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31bc509-040d-49cc-82c4-d97e64d27076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:41.354262 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d31bc509-040d-49cc-82c4-d97e64d27076 None None] MFA Rules not processed for user `b7fd6c37fc1f47af8012af87ca705a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:41.355599 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d31bc509-040d-49cc-82c4-d97e64d27076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:41.429544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:41.493468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56d8f2-0845-4c73-bdf1-848b7864d074 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:41.505128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3973c-86fb-4646-84c0-45a50caef87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:41.506771 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3973c-86fb-4646-84c0-45a50caef87c admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:41.508042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3973c-86fb-4646-84c0-45a50caef87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:41.509105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3973c-86fb-4646-84c0-45a50caef87c admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:41.510621 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3973c-86fb-4646-84c0-45a50caef87c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:41.515011 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3973c-86fb-4646-84c0-45a50caef87c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:41.544177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 954/1909] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:41 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:41.546256 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31bc509-040d-49cc-82c4-d97e64d27076 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:41.550423 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 956/1910] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:41.552326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3973c-86fb-4646-84c0-45a50caef87c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:41.556321 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cea3973c-86fb-4646-84c0-45a50caef87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:41.569978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d31bc509-040d-49cc-82c4-d97e64d27076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:41.578000 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3973c-86fb-4646-84c0-45a50caef87c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:41.584246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df48c16-feb3-4893-b392-60ca8deddb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:41.587635 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df48c16-feb3-4893-b392-60ca8deddb9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:41.588610 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df48c16-feb3-4893-b392-60ca8deddb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:41.588610 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df48c16-feb3-4893-b392-60ca8deddb9c admin admin] PATH_INFO: `/v3/projects/302f0d2ca4574f51b69a69a02f202823/users/29d386e4bfe64aeda348626dc3c315ac/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:41.596627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:41.614743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31bc509-040d-49cc-82c4-d97e64d27076 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:41.642245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aeafc9-485b-4cc5-bf2e-f8c87405a92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:41.642245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aeafc9-485b-4cc5-bf2e-f8c87405a92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:41.642652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aeafc9-485b-4cc5-bf2e-f8c87405a92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:41.643019 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aeafc9-485b-4cc5-bf2e-f8c87405a92e admin admin] PATH_INFO: `/v3/users/c520cc46a6df46ae93571116b23f56e7` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:41.644645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aeafc9-485b-4cc5-bf2e-f8c87405a92e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c520cc46a6df46ae93571116b23f56e7)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:41.646800 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aeafc9-485b-4cc5-bf2e-f8c87405a92e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:41.651229 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df48c16-feb3-4893-b392-60ca8deddb9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=302f0d2ca4574f51b69a69a02f202823, user_id=29d386e4bfe64aeda348626dc3c315ac, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:41.654349 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df48c16-feb3-4893-b392-60ca8deddb9c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:41.693283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 957/1911] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:41 2022] PUT /identity/v3/projects/302f0d2ca4574f51b69a69a02f202823/users/29d386e4bfe64aeda348626dc3c315ac/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:41.694888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df48c16-feb3-4893-b392-60ca8deddb9c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:41.705322 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0df48c16-feb3-4893-b392-60ca8deddb9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:41.817382 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-94aeafc9-485b-4cc5-bf2e-f8c87405a92e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c520cc46a6df46ae93571116b23f56e7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:41.832373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-94aeafc9-485b-4cc5-bf2e-f8c87405a92e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c520cc46a6df46ae93571116b23f56e7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:41.833402 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-94aeafc9-485b-4cc5-bf2e-f8c87405a92e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c520cc46a6df46ae93571116b23f56e7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:41.845578 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-94aeafc9-485b-4cc5-bf2e-f8c87405a92e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c520cc46a6df46ae93571116b23f56e7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:41.845876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df48c16-feb3-4893-b392-60ca8deddb9c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:41.846195 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253e873-7b23-4dcb-b9a5-14dfccaecd8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:41.847121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253e873-7b23-4dcb-b9a5-14dfccaecd8b service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:41.847423 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253e873-7b23-4dcb-b9a5-14dfccaecd8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:41.847833 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253e873-7b23-4dcb-b9a5-14dfccaecd8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:41.854896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b253e873-7b23-4dcb-b9a5-14dfccaecd8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:41.856257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-94aeafc9-485b-4cc5-bf2e-f8c87405a92e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c520cc46a6df46ae93571116b23f56e7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:41.907073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 955/1912] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:41 2022] DELETE /identity/v3/users/c520cc46a6df46ae93571116b23f56e7 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:41.911283 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa232d7-099d-40b0-b6d7-6b76f4ae8f73 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:41.912176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa232d7-099d-40b0-b6d7-6b76f4ae8f73 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:41.912970 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa232d7-099d-40b0-b6d7-6b76f4ae8f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:41.989558 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-0fa232d7-099d-40b0-b6d7-6b76f4ae8f73 None None] MFA Rules not processed for user `29d386e4bfe64aeda348626dc3c315ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:41.991965 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-0fa232d7-099d-40b0-b6d7-6b76f4ae8f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:42.014282 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253e873-7b23-4dcb-b9a5-14dfccaecd8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:42.017470 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253e873-7b23-4dcb-b9a5-14dfccaecd8b service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:42.037170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 958/1913] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:41 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:42.196329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 956/1914] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:42.201865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253e873-7b23-4dcb-b9a5-14dfccaecd8b service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:42.205454 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b253e873-7b23-4dcb-b9a5-14dfccaecd8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:42.274250 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa232d7-099d-40b0-b6d7-6b76f4ae8f73 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:42.282064 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-0fa232d7-099d-40b0-b6d7-6b76f4ae8f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:42.407217 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:42.417893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253e873-7b23-4dcb-b9a5-14dfccaecd8b service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:42.447228 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa232d7-099d-40b0-b6d7-6b76f4ae8f73 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:42.456947 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301218f8-ede4-43ac-8986-0defee8f2842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:42.458117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301218f8-ede4-43ac-8986-0defee8f2842 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:42.458647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301218f8-ede4-43ac-8986-0defee8f2842 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:42.459166 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301218f8-ede4-43ac-8986-0defee8f2842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:42.464698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-301218f8-ede4-43ac-8986-0defee8f2842 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:42.523877 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:42.636116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703d4ec-d62d-4fbc-8fa6-671e11b8cff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:42.637336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703d4ec-d62d-4fbc-8fa6-671e11b8cff3 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:42.637944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703d4ec-d62d-4fbc-8fa6-671e11b8cff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:42.638516 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703d4ec-d62d-4fbc-8fa6-671e11b8cff3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:42.639810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703d4ec-d62d-4fbc-8fa6-671e11b8cff3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:42.641268 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301218f8-ede4-43ac-8986-0defee8f2842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:42.642814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301218f8-ede4-43ac-8986-0defee8f2842 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:42.643717 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703d4ec-d62d-4fbc-8fa6-671e11b8cff3 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:42.657284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 957/1915] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:42 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:45:42.658553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301218f8-ede4-43ac-8986-0defee8f2842 service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:42.660624 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-301218f8-ede4-43ac-8986-0defee8f2842 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:42.682765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301218f8-ede4-43ac-8986-0defee8f2842 service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:42.698454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569df93-eec2-4ffc-86ce-f645aa04799b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:42.699157 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569df93-eec2-4ffc-86ce-f645aa04799b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:42.699454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569df93-eec2-4ffc-86ce-f645aa04799b admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:42.699707 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569df93-eec2-4ffc-86ce-f645aa04799b admin admin] PATH_INFO: `/v3/projects/0ddce1bc9f3845e6b4362458d5f99ca0` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:42.713787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569df93-eec2-4ffc-86ce-f645aa04799b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ddce1bc9f3845e6b4362458d5f99ca0)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:42.715261 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 959/1916] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:42 2022] POST /identity/v3/projects => generated 383 bytes in 514 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:42.720703 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703d4ec-d62d-4fbc-8fa6-671e11b8cff3 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:42.723449 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569df93-eec2-4ffc-86ce-f645aa04799b admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:42.906395 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5569df93-eec2-4ffc-86ce-f645aa04799b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ddce1bc9f3845e6b4362458d5f99ca0', 'request_id': 'req-5569df93-eec2-4ffc-86ce-f645aa04799b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:42.924315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703d4ec-d62d-4fbc-8fa6-671e11b8cff3 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:42.928759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5569df93-eec2-4ffc-86ce-f645aa04799b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ddce1bc9f3845e6b4362458d5f99ca0', 'request_id': 'req-5569df93-eec2-4ffc-86ce-f645aa04799b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:42.944006 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5569df93-eec2-4ffc-86ce-f645aa04799b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ddce1bc9f3845e6b4362458d5f99ca0', 'request_id': 'req-5569df93-eec2-4ffc-86ce-f645aa04799b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:42.982349 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5569df93-eec2-4ffc-86ce-f645aa04799b admin admin] The token cache is being invalidate because project 0ddce1bc9f3845e6b4362458d5f99ca0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:42.982771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5569df93-eec2-4ffc-86ce-f645aa04799b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:42.984071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5569df93-eec2-4ffc-86ce-f645aa04799b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:42.995547 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 958/1917] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:42 2022] DELETE /identity/v3/projects/0ddce1bc9f3845e6b4362458d5f99ca0 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:42.999112 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3d010-073b-4a7e-b67d-e04121dd497f None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:42.999112 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3d010-073b-4a7e-b67d-e04121dd497f None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:42.999728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3d010-073b-4a7e-b67d-e04121dd497f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:43.030055 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-2dc3d010-073b-4a7e-b67d-e04121dd497f None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:43.031410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2dc3d010-073b-4a7e-b67d-e04121dd497f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:43.193651 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576598a3-2be4-422c-a89d-39a129c13566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:43.194343 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576598a3-2be4-422c-a89d-39a129c13566 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:43.194626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576598a3-2be4-422c-a89d-39a129c13566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:43.198226 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576598a3-2be4-422c-a89d-39a129c13566 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:43.200632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576598a3-2be4-422c-a89d-39a129c13566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:43.209927 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576598a3-2be4-422c-a89d-39a129c13566 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:43.266408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 960/1918] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:42 2022] POST /identity/v3/users => generated 298 bytes in 547 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:43.273682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576598a3-2be4-422c-a89d-39a129c13566 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:43.276811 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-576598a3-2be4-422c-a89d-39a129c13566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:43.293453 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 959/1919] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:43.294397 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f201d-8ce3-4980-a3cd-a8ce753dd61c None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:43.309506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f201d-8ce3-4980-a3cd-a8ce753dd61c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:43.309506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f201d-8ce3-4980-a3cd-a8ce753dd61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:43.309758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576598a3-2be4-422c-a89d-39a129c13566 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:43.323145 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba193b7-00a8-4f09-95d1-40524c76f894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:43.325902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba193b7-00a8-4f09-95d1-40524c76f894 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:43.327263 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba193b7-00a8-4f09-95d1-40524c76f894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:43.328795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba193b7-00a8-4f09-95d1-40524c76f894 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:43.331657 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba193b7-00a8-4f09-95d1-40524c76f894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:43.335047 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba193b7-00a8-4f09-95d1-40524c76f894 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:43.371645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 961/1920] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:43.373338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba193b7-00a8-4f09-95d1-40524c76f894 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:43.373583 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-dc4f201d-8ce3-4980-a3cd-a8ce753dd61c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:43.384136 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-dc4f201d-8ce3-4980-a3cd-a8ce753dd61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:43.392717 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba193b7-00a8-4f09-95d1-40524c76f894 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:43.403174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d3898-dc86-4da4-ac42-8432e5b0456e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:43.404197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d3898-dc86-4da4-ac42-8432e5b0456e admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:43.404697 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d3898-dc86-4da4-ac42-8432e5b0456e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:43.405227 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d3898-dc86-4da4-ac42-8432e5b0456e admin admin] PATH_INFO: `/v3/projects/1ba51b82aba94fce85ca5e4edaeaf1de/users/92cd6a97489542c7ba90b152b23c6656/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:43.411947 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 960/1921] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:43.416477 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f201d-8ce3-4980-a3cd-a8ce753dd61c None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:43.442684 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:43.445547 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f201d-8ce3-4980-a3cd-a8ce753dd61c None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:43.466807 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d3898-dc86-4da4-ac42-8432e5b0456e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba51b82aba94fce85ca5e4edaeaf1de, user_id=92cd6a97489542c7ba90b152b23c6656, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:43.466982 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064d28c-d0ae-4226-a7a0-e65f77ddc631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:43.467307 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064d28c-d0ae-4226-a7a0-e65f77ddc631 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:43.467597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064d28c-d0ae-4226-a7a0-e65f77ddc631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:43.467888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064d28c-d0ae-4226-a7a0-e65f77ddc631 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:43.468580 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064d28c-d0ae-4226-a7a0-e65f77ddc631 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:45:43.468888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064d28c-d0ae-4226-a7a0-e65f77ddc631 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:43.470764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064d28c-d0ae-4226-a7a0-e65f77ddc631 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:43.473548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d3898-dc86-4da4-ac42-8432e5b0456e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:43.499166 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 961/1922] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:45:43.503555 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064d28c-d0ae-4226-a7a0-e65f77ddc631 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:43.508231 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 962/1923] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:43 2022] PUT /identity/v3/projects/1ba51b82aba94fce85ca5e4edaeaf1de/users/92cd6a97489542c7ba90b152b23c6656/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:43.514983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d3898-dc86-4da4-ac42-8432e5b0456e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:43.529198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064d28c-d0ae-4226-a7a0-e65f77ddc631 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:43.576720 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:43.632226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed750e-7944-4a22-bc76-34ff4c1574e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:43.632872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed750e-7944-4a22-bc76-34ff4c1574e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:43.633188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed750e-7944-4a22-bc76-34ff4c1574e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:43.634097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed750e-7944-4a22-bc76-34ff4c1574e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:43.634357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed750e-7944-4a22-bc76-34ff4c1574e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:43.636786 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed750e-7944-4a22-bc76-34ff4c1574e0 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:43.681411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 962/1924] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:43 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:45:43.686275 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed750e-7944-4a22-bc76-34ff4c1574e0 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:43.700686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d3898-dc86-4da4-ac42-8432e5b0456e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:43.818056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed750e-7944-4a22-bc76-34ff4c1574e0 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:43.829665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4399e-4a4b-411b-b5ff-93d430941ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:43.831245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4399e-4a4b-411b-b5ff-93d430941ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:43.833211 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4399e-4a4b-411b-b5ff-93d430941ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:43.833593 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4399e-4a4b-411b-b5ff-93d430941ab7 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:43.834316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4399e-4a4b-411b-b5ff-93d430941ab7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:43.836837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4399e-4a4b-411b-b5ff-93d430941ab7 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:43.883975 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 963/1925] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:43 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:45:43.891873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4399e-4a4b-411b-b5ff-93d430941ab7 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:43.895302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea68b18-ec2a-46df-9bf4-e6a1ff8946d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:43.896709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea68b18-ec2a-46df-9bf4-e6a1ff8946d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:43.898138 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea68b18-ec2a-46df-9bf4-e6a1ff8946d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:43.898647 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea68b18-ec2a-46df-9bf4-e6a1ff8946d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:43.899863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea68b18-ec2a-46df-9bf4-e6a1ff8946d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:43.908163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea68b18-ec2a-46df-9bf4-e6a1ff8946d5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:43.915243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4399e-4a4b-411b-b5ff-93d430941ab7 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:43.928382 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06142ad3-506e-429d-8991-8aa9cf693a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:43.929623 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 963/1926] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:43.934073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06142ad3-506e-429d-8991-8aa9cf693a2f admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:43.934073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06142ad3-506e-429d-8991-8aa9cf693a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:43.935087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06142ad3-506e-429d-8991-8aa9cf693a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:43.936919 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06142ad3-506e-429d-8991-8aa9cf693a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:43.939026 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea68b18-ec2a-46df-9bf4-e6a1ff8946d5 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:43.941678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06142ad3-506e-429d-8991-8aa9cf693a2f admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:43.959935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea68b18-ec2a-46df-9bf4-e6a1ff8946d5 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:43.966000 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15c6dc-b7e1-4bc6-ac34-5a19aeedb32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:43.966677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 964/1927] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:45:43.966980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15c6dc-b7e1-4bc6-ac34-5a19aeedb32e admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:43.966980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15c6dc-b7e1-4bc6-ac34-5a19aeedb32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:43.967283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15c6dc-b7e1-4bc6-ac34-5a19aeedb32e admin admin] PATH_INFO: `/v3/projects/1ba51b82aba94fce85ca5e4edaeaf1de/users/92cd6a97489542c7ba90b152b23c6656/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:43.969558 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15c6dc-b7e1-4bc6-ac34-5a19aeedb32e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba51b82aba94fce85ca5e4edaeaf1de, user_id=92cd6a97489542c7ba90b152b23c6656, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:43.971935 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06142ad3-506e-429d-8991-8aa9cf693a2f admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:43.972427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15c6dc-b7e1-4bc6-ac34-5a19aeedb32e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:44.005527 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 964/1928] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:43 2022] PUT /identity/v3/projects/1ba51b82aba94fce85ca5e4edaeaf1de/users/92cd6a97489542c7ba90b152b23c6656/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:45:44.005695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06142ad3-506e-429d-8991-8aa9cf693a2f admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:44.013973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2c393-8f1c-4d72-80f8-4a547cc86103 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:44.014132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2c393-8f1c-4d72-80f8-4a547cc86103 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:44.015203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2c393-8f1c-4d72-80f8-4a547cc86103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:44.074068 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-3cc2c393-8f1c-4d72-80f8-4a547cc86103 None None] MFA Rules not processed for user `92cd6a97489542c7ba90b152b23c6656`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:44.076627 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3cc2c393-8f1c-4d72-80f8-4a547cc86103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:44.162648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fe372-582d-4c13-9022-9b3b4522c8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:44.163587 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fe372-582d-4c13-9022-9b3b4522c8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:44.164183 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fe372-582d-4c13-9022-9b3b4522c8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:44.164510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fe372-582d-4c13-9022-9b3b4522c8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:44.165794 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fe372-582d-4c13-9022-9b3b4522c8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:44.167117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fe372-582d-4c13-9022-9b3b4522c8f4 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:44.223412 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 965/1929] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:44.223973 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fe372-582d-4c13-9022-9b3b4522c8f4 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:44.250943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fe372-582d-4c13-9022-9b3b4522c8f4 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:44.261193 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4466f0-39cb-4dcc-b1e4-3a26db344152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:44.265471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4466f0-39cb-4dcc-b1e4-3a26db344152 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:44.266009 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4466f0-39cb-4dcc-b1e4-3a26db344152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:44.266299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4466f0-39cb-4dcc-b1e4-3a26db344152 admin admin] PATH_INFO: `/v3/projects/c404486ee19f4b5eb65586527cc470aa/users/73c36de0bdbc48c5a2b11acd94fae351/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:44.292828 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 965/1930] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:44.301656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2c393-8f1c-4d72-80f8-4a547cc86103 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:44.303948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3cc2c393-8f1c-4d72-80f8-4a547cc86103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:44.305795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4466f0-39cb-4dcc-b1e4-3a26db344152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c404486ee19f4b5eb65586527cc470aa, user_id=73c36de0bdbc48c5a2b11acd94fae351, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:44.311421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4466f0-39cb-4dcc-b1e4-3a26db344152 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:44.331692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:44.337301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:44.353669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 966/1931] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:44 2022] PUT /identity/v3/projects/c404486ee19f4b5eb65586527cc470aa/users/73c36de0bdbc48c5a2b11acd94fae351/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:44.366147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfa664-2b36-40aa-ade4-a674c6eca8ec None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:44.366458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfa664-2b36-40aa-ade4-a674c6eca8ec None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:44.366994 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfa664-2b36-40aa-ade4-a674c6eca8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:44.446376 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-bebfa664-2b36-40aa-ade4-a674c6eca8ec None None] MFA Rules not processed for user `73c36de0bdbc48c5a2b11acd94fae351`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:44.448767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-bebfa664-2b36-40aa-ade4-a674c6eca8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:44.483576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2c393-8f1c-4d72-80f8-4a547cc86103 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:44.594545 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 967/1932] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:44.645342 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96374972-4cfe-4ab4-aac8-5547698b068c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:44.646096 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96374972-4cfe-4ab4-aac8-5547698b068c service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:44.646448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96374972-4cfe-4ab4-aac8-5547698b068c service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:44.646792 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96374972-4cfe-4ab4-aac8-5547698b068c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:44.650412 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-96374972-4cfe-4ab4-aac8-5547698b068c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:44.677695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfa664-2b36-40aa-ade4-a674c6eca8ec None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:44.677695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-bebfa664-2b36-40aa-ade4-a674c6eca8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:44.688066 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfa664-2b36-40aa-ade4-a674c6eca8ec None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:44.695857 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba55f43-35cc-4219-a691-5b5452ccdfb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:44.697721 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba55f43-35cc-4219-a691-5b5452ccdfb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:44.698053 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba55f43-35cc-4219-a691-5b5452ccdfb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:44.698322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba55f43-35cc-4219-a691-5b5452ccdfb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:44.718162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba55f43-35cc-4219-a691-5b5452ccdfb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:44.722097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba55f43-35cc-4219-a691-5b5452ccdfb1 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:44.761363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 968/1933] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:45:44.817113 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96374972-4cfe-4ab4-aac8-5547698b068c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:44.820241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96374972-4cfe-4ab4-aac8-5547698b068c service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:44.843582 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 966/1934] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:47.831894 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba55f43-35cc-4219-a691-5b5452ccdfb1 service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:47.847403 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba55f43-35cc-4219-a691-5b5452ccdfb1 service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:47.853626 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d5afe-90f2-4237-8bc6-65e0d34c8756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:47.854693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d5afe-90f2-4237-8bc6-65e0d34c8756 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:47.855225 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d5afe-90f2-4237-8bc6-65e0d34c8756 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:47.855717 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d5afe-90f2-4237-8bc6-65e0d34c8756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:47.859336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c62d5afe-90f2-4237-8bc6-65e0d34c8756 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:47.931136 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d5afe-90f2-4237-8bc6-65e0d34c8756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:47.933016 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d5afe-90f2-4237-8bc6-65e0d34c8756 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:47.945032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 969/1935] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:51.437538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96374972-4cfe-4ab4-aac8-5547698b068c service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:51.447668 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96374972-4cfe-4ab4-aac8-5547698b068c service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:51.451791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e0e14-6c93-43bf-940f-93259a039620 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:51.452456 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e0e14-6c93-43bf-940f-93259a039620 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:51.452717 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e0e14-6c93-43bf-940f-93259a039620 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:51.452986 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e0e14-6c93-43bf-940f-93259a039620 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:51.519165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e0e14-6c93-43bf-940f-93259a039620 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:51.520727 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e0e14-6c93-43bf-940f-93259a039620 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:51.531770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 967/1936] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:45:52.666812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d5afe-90f2-4237-8bc6-65e0d34c8756 service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:52.669341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c62d5afe-90f2-4237-8bc6-65e0d34c8756 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:52.781114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d5afe-90f2-4237-8bc6-65e0d34c8756 service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:52.789339 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130804f-9047-4a81-819e-27fd14c51c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:52.791091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130804f-9047-4a81-819e-27fd14c51c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:52.791809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130804f-9047-4a81-819e-27fd14c51c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:52.792348 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130804f-9047-4a81-819e-27fd14c51c8d admin admin] PATH_INFO: `/v3/users/e45671b12f6f4284adfa0a3d1777f1e3` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:52.794441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130804f-9047-4a81-819e-27fd14c51c8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e45671b12f6f4284adfa0a3d1777f1e3)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:52.799105 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130804f-9047-4a81-819e-27fd14c51c8d admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:52.836025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:52.896887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c130804f-9047-4a81-819e-27fd14c51c8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e45671b12f6f4284adfa0a3d1777f1e3'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:52.910515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c130804f-9047-4a81-819e-27fd14c51c8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e45671b12f6f4284adfa0a3d1777f1e3'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:52.911504 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c130804f-9047-4a81-819e-27fd14c51c8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e45671b12f6f4284adfa0a3d1777f1e3'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:52.912766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c130804f-9047-4a81-819e-27fd14c51c8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e45671b12f6f4284adfa0a3d1777f1e3'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:52.926741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c130804f-9047-4a81-819e-27fd14c51c8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e45671b12f6f4284adfa0a3d1777f1e3'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:52.959688 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 970/1937] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:52 2022] DELETE /identity/v3/users/e45671b12f6f4284adfa0a3d1777f1e3 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:52.966660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96bbd6-bef4-4d35-a921-cd06bc1ec9ac None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:52.967565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96bbd6-bef4-4d35-a921-cd06bc1ec9ac None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:52.967565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96bbd6-bef4-4d35-a921-cd06bc1ec9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:53.005117 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8f96bbd6-bef4-4d35-a921-cd06bc1ec9ac None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:53.007463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8f96bbd6-bef4-4d35-a921-cd06bc1ec9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:53.015779 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:53.138838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 968/1938] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:45:53.159934 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130804f-9047-4a81-819e-27fd14c51c8d admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:53.164257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c130804f-9047-4a81-819e-27fd14c51c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:53.274969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130804f-9047-4a81-819e-27fd14c51c8d admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:53.279917 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee244e-82e7-4ae4-bace-447422785ec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:53.280960 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee244e-82e7-4ae4-bace-447422785ec9 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:53.281699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee244e-82e7-4ae4-bace-447422785ec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:53.282235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee244e-82e7-4ae4-bace-447422785ec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:53.297369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee244e-82e7-4ae4-bace-447422785ec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:53.299562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee244e-82e7-4ae4-bace-447422785ec9 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:53.349783 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 971/1939] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:53.793014 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96bbd6-bef4-4d35-a921-cd06bc1ec9ac None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:53.795160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8f96bbd6-bef4-4d35-a921-cd06bc1ec9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:53.816508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96bbd6-bef4-4d35-a921-cd06bc1ec9ac None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:53.823692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0950e38-8833-4de2-b5c9-9e82ec5be19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:53.824505 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0950e38-8833-4de2-b5c9-9e82ec5be19f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:53.824881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0950e38-8833-4de2-b5c9-9e82ec5be19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:53.825296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0950e38-8833-4de2-b5c9-9e82ec5be19f admin admin] PATH_INFO: `/v3/projects/e9938011d6254805be9c8b15d02be673` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:53.826831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0950e38-8833-4de2-b5c9-9e82ec5be19f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9938011d6254805be9c8b15d02be673)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:53.829987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0950e38-8833-4de2-b5c9-9e82ec5be19f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:53.899155 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c0950e38-8833-4de2-b5c9-9e82ec5be19f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9938011d6254805be9c8b15d02be673', 'request_id': 'req-c0950e38-8833-4de2-b5c9-9e82ec5be19f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:53.906323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c0950e38-8833-4de2-b5c9-9e82ec5be19f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9938011d6254805be9c8b15d02be673', 'request_id': 'req-c0950e38-8833-4de2-b5c9-9e82ec5be19f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:53.916157 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c0950e38-8833-4de2-b5c9-9e82ec5be19f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9938011d6254805be9c8b15d02be673', 'request_id': 'req-c0950e38-8833-4de2-b5c9-9e82ec5be19f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:53.932445 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c0950e38-8833-4de2-b5c9-9e82ec5be19f admin admin] The token cache is being invalidate because project e9938011d6254805be9c8b15d02be673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:53.932824 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c0950e38-8833-4de2-b5c9-9e82ec5be19f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:53.933551 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c0950e38-8833-4de2-b5c9-9e82ec5be19f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:53.935752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 969/1940] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:53 2022] DELETE /identity/v3/projects/e9938011d6254805be9c8b15d02be673 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:53.940633 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee244e-82e7-4ae4-bace-447422785ec9 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:53.951429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee244e-82e7-4ae4-bace-447422785ec9 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:54.021219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446cb27-1696-4a1e-967a-290491a2cd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:54.021928 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446cb27-1696-4a1e-967a-290491a2cd79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:54.022228 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446cb27-1696-4a1e-967a-290491a2cd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:54.022523 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446cb27-1696-4a1e-967a-290491a2cd79 admin admin] PATH_INFO: `/v3/users/b7fd6c37fc1f47af8012af87ca705a5a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:54.023768 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446cb27-1696-4a1e-967a-290491a2cd79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7fd6c37fc1f47af8012af87ca705a5a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:54.025869 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446cb27-1696-4a1e-967a-290491a2cd79 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:54.085715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:54.102627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-e446cb27-1696-4a1e-967a-290491a2cd79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7fd6c37fc1f47af8012af87ca705a5a'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:54.115242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-e446cb27-1696-4a1e-967a-290491a2cd79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7fd6c37fc1f47af8012af87ca705a5a'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:54.116050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-e446cb27-1696-4a1e-967a-290491a2cd79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7fd6c37fc1f47af8012af87ca705a5a'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:54.116949 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-e446cb27-1696-4a1e-967a-290491a2cd79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7fd6c37fc1f47af8012af87ca705a5a'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:54.129798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-e446cb27-1696-4a1e-967a-290491a2cd79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7fd6c37fc1f47af8012af87ca705a5a'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:54.158853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 972/1941] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:53 2022] DELETE /identity/v3/users/b7fd6c37fc1f47af8012af87ca705a5a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:54.673401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0950e38-8833-4de2-b5c9-9e82ec5be19f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:54.678183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c0950e38-8833-4de2-b5c9-9e82ec5be19f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:54.709896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0950e38-8833-4de2-b5c9-9e82ec5be19f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:54.803475 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456a3e0-8a32-4727-8773-1e1c4b69905b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:54.804143 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456a3e0-8a32-4727-8773-1e1c4b69905b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:54.804423 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456a3e0-8a32-4727-8773-1e1c4b69905b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:54.804699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456a3e0-8a32-4727-8773-1e1c4b69905b admin admin] PATH_INFO: `/v3/projects/95149a85bf3749f0b8c53722656c3a27` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:54.806080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456a3e0-8a32-4727-8773-1e1c4b69905b admin admin] RBAC: Authorizing `identity:delete_project(project_id=95149a85bf3749f0b8c53722656c3a27)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:54.808364 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456a3e0-8a32-4727-8773-1e1c4b69905b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:54.857923 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:54.873247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6456a3e0-8a32-4727-8773-1e1c4b69905b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95149a85bf3749f0b8c53722656c3a27', 'request_id': 'req-6456a3e0-8a32-4727-8773-1e1c4b69905b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:54.881985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6456a3e0-8a32-4727-8773-1e1c4b69905b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95149a85bf3749f0b8c53722656c3a27', 'request_id': 'req-6456a3e0-8a32-4727-8773-1e1c4b69905b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:54.891958 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6456a3e0-8a32-4727-8773-1e1c4b69905b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95149a85bf3749f0b8c53722656c3a27', 'request_id': 'req-6456a3e0-8a32-4727-8773-1e1c4b69905b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:54.908583 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6456a3e0-8a32-4727-8773-1e1c4b69905b admin admin] The token cache is being invalidate because project 95149a85bf3749f0b8c53722656c3a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:54.908881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6456a3e0-8a32-4727-8773-1e1c4b69905b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:54.909633 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6456a3e0-8a32-4727-8773-1e1c4b69905b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:54.912000 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 970/1942] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:54 2022] DELETE /identity/v3/projects/95149a85bf3749f0b8c53722656c3a27 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:54.919531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8c11e-cb70-4b12-bcc6-357d73e9bbe9 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:54.920021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8c11e-cb70-4b12-bcc6-357d73e9bbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:54.920209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8c11e-cb70-4b12-bcc6-357d73e9bbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:54.945844 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-c2f8c11e-cb70-4b12-bcc6-357d73e9bbe9 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:54.947410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c2f8c11e-cb70-4b12-bcc6-357d73e9bbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:55.059172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 973/1943] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:55.069305 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be2d42-9958-45c1-94f7-9b1208c77dfc None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:55.069667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be2d42-9958-45c1-94f7-9b1208c77dfc None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:55.069992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be2d42-9958-45c1-94f7-9b1208c77dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:55.095094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-38be2d42-9958-45c1-94f7-9b1208c77dfc None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:55.096397 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-38be2d42-9958-45c1-94f7-9b1208c77dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:55.106345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 971/1944] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:55.111187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8c11e-cb70-4b12-bcc6-357d73e9bbe9 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:55.126347 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8c11e-cb70-4b12-bcc6-357d73e9bbe9 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:55.133274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8c79b-13cb-4b3e-a958-ded13af85f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:55.134098 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8c79b-13cb-4b3e-a958-ded13af85f91 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:55.134613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8c79b-13cb-4b3e-a958-ded13af85f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:55.135010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8c79b-13cb-4b3e-a958-ded13af85f91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:55.135815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8c79b-13cb-4b3e-a958-ded13af85f91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:45:55.136223 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8c79b-13cb-4b3e-a958-ded13af85f91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:55.138144 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8c79b-13cb-4b3e-a958-ded13af85f91 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:55.153308 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 974/1945] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:45:55.158681 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be2d42-9958-45c1-94f7-9b1208c77dfc None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:55.170051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be2d42-9958-45c1-94f7-9b1208c77dfc None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:55.176413 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1425da-073e-4d58-96f7-6826141e97c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:55.177278 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1425da-073e-4d58-96f7-6826141e97c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:55.177693 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1425da-073e-4d58-96f7-6826141e97c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:55.178101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1425da-073e-4d58-96f7-6826141e97c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:55.178915 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1425da-073e-4d58-96f7-6826141e97c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:55.181032 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1425da-073e-4d58-96f7-6826141e97c8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:55.237169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 972/1946] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:55 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:55.250292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8c79b-13cb-4b3e-a958-ded13af85f91 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:55.263781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8c79b-13cb-4b3e-a958-ded13af85f91 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:55.358349 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b488e5-d280-405d-a3a4-ea5b04f5eeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:55.358895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b488e5-d280-405d-a3a4-ea5b04f5eeb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:55.359174 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b488e5-d280-405d-a3a4-ea5b04f5eeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:55.359434 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b488e5-d280-405d-a3a4-ea5b04f5eeb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:55.360098 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b488e5-d280-405d-a3a4-ea5b04f5eeb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:55.362175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b488e5-d280-405d-a3a4-ea5b04f5eeb0 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:55.392017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 975/1947] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:55 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:55.396817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1425da-073e-4d58-96f7-6826141e97c8 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:55.413869 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1425da-073e-4d58-96f7-6826141e97c8 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:55.423658 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cc246-16ae-43bf-8a49-8a5cb5d91b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:55.424536 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cc246-16ae-43bf-8a49-8a5cb5d91b77 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:55.425020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cc246-16ae-43bf-8a49-8a5cb5d91b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:55.425486 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cc246-16ae-43bf-8a49-8a5cb5d91b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:55.426830 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019cc246-16ae-43bf-8a49-8a5cb5d91b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:55.429928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019cc246-16ae-43bf-8a49-8a5cb5d91b77 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:55.459641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 973/1948] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:45:55.462621 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b488e5-d280-405d-a3a4-ea5b04f5eeb0 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:55.503271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b488e5-d280-405d-a3a4-ea5b04f5eeb0 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:55.517319 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d33c12-d7ec-479d-aa06-a097d98aff70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:55.517957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d33c12-d7ec-479d-aa06-a097d98aff70 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:55.518601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d33c12-d7ec-479d-aa06-a097d98aff70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:55.518872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d33c12-d7ec-479d-aa06-a097d98aff70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:55.519650 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d33c12-d7ec-479d-aa06-a097d98aff70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:55.521359 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d33c12-d7ec-479d-aa06-a097d98aff70 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:55.542254 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 976/1949] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:55.544158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cc246-16ae-43bf-8a49-8a5cb5d91b77 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:55.564211 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cc246-16ae-43bf-8a49-8a5cb5d91b77 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:55.571785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f235219-a3ef-4c5b-bb56-9165f03506a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:55.572615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f235219-a3ef-4c5b-bb56-9165f03506a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:55.573037 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f235219-a3ef-4c5b-bb56-9165f03506a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:55.573449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f235219-a3ef-4c5b-bb56-9165f03506a1 admin admin] PATH_INFO: `/v3/projects/2251a89cc2a74e55895b6b9b1618b802/users/9871b09c5b424746af3c19bcde6a0670/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:55.615344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f235219-a3ef-4c5b-bb56-9165f03506a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2251a89cc2a74e55895b6b9b1618b802, user_id=9871b09c5b424746af3c19bcde6a0670, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:55.619069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f235219-a3ef-4c5b-bb56-9165f03506a1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:55.657751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 974/1950] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:55 2022] PUT /identity/v3/projects/2251a89cc2a74e55895b6b9b1618b802/users/9871b09c5b424746af3c19bcde6a0670/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:45:55.662727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542e3dd-94a2-496a-ac90-4bdf0cb64930 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:55.663178 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542e3dd-94a2-496a-ac90-4bdf0cb64930 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:55.663827 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542e3dd-94a2-496a-ac90-4bdf0cb64930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:55.702700 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-0542e3dd-94a2-496a-ac90-4bdf0cb64930 None None] MFA Rules not processed for user `9871b09c5b424746af3c19bcde6a0670`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:55.704301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-0542e3dd-94a2-496a-ac90-4bdf0cb64930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:55.717038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:55.843098 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 977/1951] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:55.853072 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f235219-a3ef-4c5b-bb56-9165f03506a1 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:55.959528 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f235219-a3ef-4c5b-bb56-9165f03506a1 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:55.965626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf379d6-a609-4026-b675-ff7b739bd65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:55.966434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf379d6-a609-4026-b675-ff7b739bd65b admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:55.968171 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf379d6-a609-4026-b675-ff7b739bd65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:55.968171 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf379d6-a609-4026-b675-ff7b739bd65b admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:55.968415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf379d6-a609-4026-b675-ff7b739bd65b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:55.970531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf379d6-a609-4026-b675-ff7b739bd65b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:56.031826 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 975/1952] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:55 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:56.034691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542e3dd-94a2-496a-ac90-4bdf0cb64930 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:56.053093 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542e3dd-94a2-496a-ac90-4bdf0cb64930 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:56.160462 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63b677-e378-4f46-8b8d-04bb586c6753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:56.160983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63b677-e378-4f46-8b8d-04bb586c6753 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:56.161310 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63b677-e378-4f46-8b8d-04bb586c6753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:56.161594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63b677-e378-4f46-8b8d-04bb586c6753 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:56.162566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63b677-e378-4f46-8b8d-04bb586c6753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:56.164544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63b677-e378-4f46-8b8d-04bb586c6753 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:56.202435 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 978/1953] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:56 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:56.204650 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf379d6-a609-4026-b675-ff7b739bd65b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:56.229667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf379d6-a609-4026-b675-ff7b739bd65b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:56.248180 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b60b5-b63d-432e-8e35-a0d2e0af5c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:56.249638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b60b5-b63d-432e-8e35-a0d2e0af5c86 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:56.250245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b60b5-b63d-432e-8e35-a0d2e0af5c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:56.250746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b60b5-b63d-432e-8e35-a0d2e0af5c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:56.252456 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b60b5-b63d-432e-8e35-a0d2e0af5c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:56.256837 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b60b5-b63d-432e-8e35-a0d2e0af5c86 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:56.289335 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 976/1954] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:56.291862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63b677-e378-4f46-8b8d-04bb586c6753 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:56.307261 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63b677-e378-4f46-8b8d-04bb586c6753 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:56.317460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd7424c-39e1-4f40-bee6-365da7e790e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:56.318250 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7424c-39e1-4f40-bee6-365da7e790e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:56.318824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7424c-39e1-4f40-bee6-365da7e790e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:56.319147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7424c-39e1-4f40-bee6-365da7e790e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:56.320014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd7424c-39e1-4f40-bee6-365da7e790e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:56.322382 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd7424c-39e1-4f40-bee6-365da7e790e5 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:56.344224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 979/1955] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:45:56.345604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b60b5-b63d-432e-8e35-a0d2e0af5c86 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:56.370982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b60b5-b63d-432e-8e35-a0d2e0af5c86 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:56.380233 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff83b5d-6bd6-4f70-bbf9-c4fadcecb2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:56.381439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff83b5d-6bd6-4f70-bbf9-c4fadcecb2ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:56.381965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff83b5d-6bd6-4f70-bbf9-c4fadcecb2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:56.382465 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff83b5d-6bd6-4f70-bbf9-c4fadcecb2ca admin admin] PATH_INFO: `/v3/projects/fc7f5fbf7a2d491f9550117c2f3f4f73/users/bdd5d3d4eb154acca5eb65f4a1048308/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:56.412098 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff83b5d-6bd6-4f70-bbf9-c4fadcecb2ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc7f5fbf7a2d491f9550117c2f3f4f73, user_id=bdd5d3d4eb154acca5eb65f4a1048308, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:56.414859 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff83b5d-6bd6-4f70-bbf9-c4fadcecb2ca admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:56.442759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 977/1956] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:56 2022] PUT /identity/v3/projects/fc7f5fbf7a2d491f9550117c2f3f4f73/users/bdd5d3d4eb154acca5eb65f4a1048308/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:45:56.459817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b284a-9aa6-4ff2-8a52-37f00c20b03b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:56.460102 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b284a-9aa6-4ff2-8a52-37f00c20b03b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:56.460457 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b284a-9aa6-4ff2-8a52-37f00c20b03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:56.502462 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-9f5b284a-9aa6-4ff2-8a52-37f00c20b03b None None] MFA Rules not processed for user `bdd5d3d4eb154acca5eb65f4a1048308`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:56.504267 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9f5b284a-9aa6-4ff2-8a52-37f00c20b03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:56.609415 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 980/1957] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:56.616536 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff83b5d-6bd6-4f70-bbf9-c4fadcecb2ca admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:56.650356 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:56.789416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff83b5d-6bd6-4f70-bbf9-c4fadcecb2ca admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:56.799973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb84eb-59ed-428a-9854-48978691a264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:56.799973 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb84eb-59ed-428a-9854-48978691a264 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:56.800760 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb84eb-59ed-428a-9854-48978691a264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:56.801328 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb84eb-59ed-428a-9854-48978691a264 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:56.802722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb84eb-59ed-428a-9854-48978691a264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:56.807016 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb84eb-59ed-428a-9854-48978691a264 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:56.838323 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b284a-9aa6-4ff2-8a52-37f00c20b03b None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:56.840756 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9f5b284a-9aa6-4ff2-8a52-37f00c20b03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:56.867909 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 978/1958] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:56 2022] POST /identity/v3/projects => generated 379 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:56.869801 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb84eb-59ed-428a-9854-48978691a264 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:56.872635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b284a-9aa6-4ff2-8a52-37f00c20b03b None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:56.912461 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:56.966742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb84eb-59ed-428a-9854-48978691a264 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:56.973062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1bee2-2430-4e3c-a909-50314c34878b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:56.974052 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1bee2-2430-4e3c-a909-50314c34878b admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:56.975603 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1bee2-2430-4e3c-a909-50314c34878b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:56.976146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1bee2-2430-4e3c-a909-50314c34878b admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:56.977008 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1bee2-2430-4e3c-a909-50314c34878b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:56.979322 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1bee2-2430-4e3c-a909-50314c34878b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:57.016938 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 979/1959] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:56 2022] POST /identity/v3/users => generated 296 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:57.019344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1bee2-2430-4e3c-a909-50314c34878b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:57.034414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1bee2-2430-4e3c-a909-50314c34878b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:57.041283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee60e7-4493-4d77-b7de-5e31104f1c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:57.043133 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee60e7-4493-4d77-b7de-5e31104f1c7d admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:57.043610 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee60e7-4493-4d77-b7de-5e31104f1c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:57.044086 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee60e7-4493-4d77-b7de-5e31104f1c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:57.045287 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee60e7-4493-4d77-b7de-5e31104f1c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:57.051980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70903d4d-5294-4a2e-8b60-c2c3980a1198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:57.051980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70903d4d-5294-4a2e-8b60-c2c3980a1198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:57.051980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70903d4d-5294-4a2e-8b60-c2c3980a1198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:57.052839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee60e7-4493-4d77-b7de-5e31104f1c7d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:57.052935 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70903d4d-5294-4a2e-8b60-c2c3980a1198 admin admin] PATH_INFO: `/v3/users/73c36de0bdbc48c5a2b11acd94fae351` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:57.054487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70903d4d-5294-4a2e-8b60-c2c3980a1198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c36de0bdbc48c5a2b11acd94fae351)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:57.056449 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70903d4d-5294-4a2e-8b60-c2c3980a1198 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:57.082451 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 980/1960] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:57.089531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee60e7-4493-4d77-b7de-5e31104f1c7d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:57.109229 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee60e7-4493-4d77-b7de-5e31104f1c7d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:57.119888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca2c39-d668-4fa5-a3e1-a69f1fc7c533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:57.121302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca2c39-d668-4fa5-a3e1-a69f1fc7c533 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:57.121992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca2c39-d668-4fa5-a3e1-a69f1fc7c533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:57.123966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca2c39-d668-4fa5-a3e1-a69f1fc7c533 admin admin] PATH_INFO: `/v3/projects/10a87b1feaea474ab429df8fee9a62a0/users/79820bbdeae24db69d975f0d3468cf5b/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:57.152243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-70903d4d-5294-4a2e-8b60-c2c3980a1198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c36de0bdbc48c5a2b11acd94fae351'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:57.161506 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca2c39-d668-4fa5-a3e1-a69f1fc7c533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10a87b1feaea474ab429df8fee9a62a0, user_id=79820bbdeae24db69d975f0d3468cf5b, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:57.167963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca2c39-d668-4fa5-a3e1-a69f1fc7c533 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:57.177946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-70903d4d-5294-4a2e-8b60-c2c3980a1198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c36de0bdbc48c5a2b11acd94fae351'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:57.178865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-70903d4d-5294-4a2e-8b60-c2c3980a1198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c36de0bdbc48c5a2b11acd94fae351'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:57.179708 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-70903d4d-5294-4a2e-8b60-c2c3980a1198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c36de0bdbc48c5a2b11acd94fae351'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:57.191034 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-70903d4d-5294-4a2e-8b60-c2c3980a1198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c36de0bdbc48c5a2b11acd94fae351'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:57.204263 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 981/1961] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:57 2022] PUT /identity/v3/projects/10a87b1feaea474ab429df8fee9a62a0/users/79820bbdeae24db69d975f0d3468cf5b/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:57.212165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca2c39-d668-4fa5-a3e1-a69f1fc7c533 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:57.223062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-32ca2c39-d668-4fa5-a3e1-a69f1fc7c533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:57.249079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 981/1962] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:56 2022] DELETE /identity/v3/users/73c36de0bdbc48c5a2b11acd94fae351 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:57.251058 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70903d4d-5294-4a2e-8b60-c2c3980a1198 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:57.253044 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-70903d4d-5294-4a2e-8b60-c2c3980a1198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:57.323980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:57.343573 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70903d4d-5294-4a2e-8b60-c2c3980a1198 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:57.345628 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca2c39-d668-4fa5-a3e1-a69f1fc7c533 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:57.349121 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4eccc8-93db-4717-9b13-e8ed963f2864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:57.349824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4eccc8-93db-4717-9b13-e8ed963f2864 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:57.350133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4eccc8-93db-4717-9b13-e8ed963f2864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:57.350395 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4eccc8-93db-4717-9b13-e8ed963f2864 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:57.351093 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4eccc8-93db-4717-9b13-e8ed963f2864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:57.352402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217a404-6cd2-43fc-a819-cb2be929ce53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:57.353078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4eccc8-93db-4717-9b13-e8ed963f2864 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:57.353310 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217a404-6cd2-43fc-a819-cb2be929ce53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:57.353773 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217a404-6cd2-43fc-a819-cb2be929ce53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:57.354243 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217a404-6cd2-43fc-a819-cb2be929ce53 admin admin] PATH_INFO: `/v3/users/bda11de729f0434aacc89f9d23c705ff` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:57.355980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217a404-6cd2-43fc-a819-cb2be929ce53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bda11de729f0434aacc89f9d23c705ff)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:57.358677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217a404-6cd2-43fc-a819-cb2be929ce53 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:57.368364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 982/1963] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:57.368364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090e527-b419-41f4-9e46-22e0ead590c6 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:57.368364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090e527-b419-41f4-9e46-22e0ead590c6 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:57.368364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090e527-b419-41f4-9e46-22e0ead590c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:57.390590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-b090e527-b419-41f4-9e46-22e0ead590c6 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:57.395331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-b090e527-b419-41f4-9e46-22e0ead590c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:57.408793 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 983/1964] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:57.415199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090e527-b419-41f4-9e46-22e0ead590c6 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:57.431226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090e527-b419-41f4-9e46-22e0ead590c6 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:57.432872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a217a404-6cd2-43fc-a819-cb2be929ce53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bda11de729f0434aacc89f9d23c705ff'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:57.437211 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea39e20-b580-40b9-a774-00bbf725c48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:57.438022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea39e20-b580-40b9-a774-00bbf725c48e admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:57.438341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea39e20-b580-40b9-a774-00bbf725c48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:57.438622 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea39e20-b580-40b9-a774-00bbf725c48e admin admin] PATH_INFO: `/v3/projects/10a87b1feaea474ab429df8fee9a62a0/users/79820bbdeae24db69d975f0d3468cf5b/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:57.441022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea39e20-b580-40b9-a774-00bbf725c48e admin admin] RBAC: Authorizing `identity:create_grant(project_id=10a87b1feaea474ab429df8fee9a62a0, user_id=79820bbdeae24db69d975f0d3468cf5b, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:57.443487 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a217a404-6cd2-43fc-a819-cb2be929ce53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bda11de729f0434aacc89f9d23c705ff'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:57.444229 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea39e20-b580-40b9-a774-00bbf725c48e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:57.444355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a217a404-6cd2-43fc-a819-cb2be929ce53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bda11de729f0434aacc89f9d23c705ff'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:57.445016 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a217a404-6cd2-43fc-a819-cb2be929ce53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bda11de729f0434aacc89f9d23c705ff'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:57.466715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 984/1965] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:57 2022] PUT /identity/v3/projects/10a87b1feaea474ab429df8fee9a62a0/users/79820bbdeae24db69d975f0d3468cf5b/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:57.466715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea39e20-b580-40b9-a774-00bbf725c48e admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:57.469717 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-cea39e20-b580-40b9-a774-00bbf725c48e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:57.474851 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a217a404-6cd2-43fc-a819-cb2be929ce53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bda11de729f0434aacc89f9d23c705ff'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:57.490723 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 982/1966] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:57 2022] DELETE /identity/v3/users/bda11de729f0434aacc89f9d23c705ff => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:57.493782 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:57.495004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d544e93-cc75-428a-bba1-414658f2a981 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:57.495365 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d544e93-cc75-428a-bba1-414658f2a981 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:57.495768 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d544e93-cc75-428a-bba1-414658f2a981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:57.528221 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-6d544e93-cc75-428a-bba1-414658f2a981 None None] MFA Rules not processed for user `79820bbdeae24db69d975f0d3468cf5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:57.529704 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6d544e93-cc75-428a-bba1-414658f2a981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:57.556621 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea39e20-b580-40b9-a774-00bbf725c48e admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:57.617750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555d981-cde7-473f-adab-989fca584a58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:57.618685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555d981-cde7-473f-adab-989fca584a58 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:57.619173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555d981-cde7-473f-adab-989fca584a58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:57.619664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555d981-cde7-473f-adab-989fca584a58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:57.621855 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2555d981-cde7-473f-adab-989fca584a58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:57.643653 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 983/1967] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:57.650750 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d4074-9a90-4990-9434-965e0f7e456f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:57.651047 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d4074-9a90-4990-9434-965e0f7e456f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:57.651317 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d4074-9a90-4990-9434-965e0f7e456f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:57.682187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-6a9d4074-9a90-4990-9434-965e0f7e456f None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:57.683448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6a9d4074-9a90-4990-9434-965e0f7e456f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:57.722289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555d981-cde7-473f-adab-989fca584a58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:57.723846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555d981-cde7-473f-adab-989fca584a58 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:57.736784 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 985/1968] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:57.737758 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555d981-cde7-473f-adab-989fca584a58 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:57.739593 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2555d981-cde7-473f-adab-989fca584a58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:57.763841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:57.784094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 984/1969] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:57.806539 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d4074-9a90-4990-9434-965e0f7e456f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:57.814447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6a9d4074-9a90-4990-9434-965e0f7e456f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:57.826912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555d981-cde7-473f-adab-989fca584a58 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:57.831754 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b070e7-49df-4e11-ba17-107bf238d288 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:57.832449 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b070e7-49df-4e11-ba17-107bf238d288 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:57.832727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b070e7-49df-4e11-ba17-107bf238d288 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:57.833025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b070e7-49df-4e11-ba17-107bf238d288 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:57.843435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d4074-9a90-4990-9434-965e0f7e456f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:57.845036 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b070e7-49df-4e11-ba17-107bf238d288 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:57.846535 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b070e7-49df-4e11-ba17-107bf238d288 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:57.852101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db999e40-a644-4b1a-864d-b205d2c4906c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:57.853113 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db999e40-a644-4b1a-864d-b205d2c4906c service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:57.854713 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db999e40-a644-4b1a-864d-b205d2c4906c service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:57.855176 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db999e40-a644-4b1a-864d-b205d2c4906c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:57.859373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 986/1970] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:57.879692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db999e40-a644-4b1a-864d-b205d2c4906c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:57.884492 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db999e40-a644-4b1a-864d-b205d2c4906c service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:57.903353 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 985/1971] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:57.925958 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b070e7-49df-4e11-ba17-107bf238d288 service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:57.927442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-a2b070e7-49df-4e11-ba17-107bf238d288 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:57.952808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b070e7-49df-4e11-ba17-107bf238d288 service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:57.961200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa118c-ed77-42ec-8749-828dedeb9363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:57.961946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa118c-ed77-42ec-8749-828dedeb9363 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:57.962599 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa118c-ed77-42ec-8749-828dedeb9363 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:57.963219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa118c-ed77-42ec-8749-828dedeb9363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:57.966356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-aeaa118c-ed77-42ec-8749-828dedeb9363 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:58.017697 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:58.113692 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa118c-ed77-42ec-8749-828dedeb9363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:58.117828 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa118c-ed77-42ec-8749-828dedeb9363 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:58.141451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 987/1972] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:45:58.340731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db999e40-a644-4b1a-864d-b205d2c4906c service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:58.341457 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-db999e40-a644-4b1a-864d-b205d2c4906c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:58.404009 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db999e40-a644-4b1a-864d-b205d2c4906c service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:58.414525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bab6ed-949f-4bb0-abea-13eea4799dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:58.415696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bab6ed-949f-4bb0-abea-13eea4799dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:58.415696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bab6ed-949f-4bb0-abea-13eea4799dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:58.415885 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bab6ed-949f-4bb0-abea-13eea4799dc9 admin admin] PATH_INFO: `/v3/projects/4f484d66ba0b47e5a9c04d6f2b1bf88c` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:58.417024 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bab6ed-949f-4bb0-abea-13eea4799dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f484d66ba0b47e5a9c04d6f2b1bf88c)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:58.419394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bab6ed-949f-4bb0-abea-13eea4799dc9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:58.508834 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-44bab6ed-949f-4bb0-abea-13eea4799dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f484d66ba0b47e5a9c04d6f2b1bf88c', 'request_id': 'req-44bab6ed-949f-4bb0-abea-13eea4799dc9'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:58.517096 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa118c-ed77-42ec-8749-828dedeb9363 service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:58.519450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-aeaa118c-ed77-42ec-8749-828dedeb9363 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:58.529267 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-44bab6ed-949f-4bb0-abea-13eea4799dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f484d66ba0b47e5a9c04d6f2b1bf88c', 'request_id': 'req-44bab6ed-949f-4bb0-abea-13eea4799dc9'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:58.547874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-44bab6ed-949f-4bb0-abea-13eea4799dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f484d66ba0b47e5a9c04d6f2b1bf88c', 'request_id': 'req-44bab6ed-949f-4bb0-abea-13eea4799dc9'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:58.548536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa118c-ed77-42ec-8749-828dedeb9363 service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:58.575564 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-44bab6ed-949f-4bb0-abea-13eea4799dc9 admin admin] The token cache is being invalidate because project 4f484d66ba0b47e5a9c04d6f2b1bf88c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:58.576427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-44bab6ed-949f-4bb0-abea-13eea4799dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:58.577761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-44bab6ed-949f-4bb0-abea-13eea4799dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:58.585391 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 986/1973] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:58 2022] DELETE /identity/v3/projects/4f484d66ba0b47e5a9c04d6f2b1bf88c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:58.587783 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bab6ed-949f-4bb0-abea-13eea4799dc9 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:58.592655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-44bab6ed-949f-4bb0-abea-13eea4799dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:58.656061 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5806d-654f-4424-b028-485643be83f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:58.657204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5806d-654f-4424-b028-485643be83f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:58.658329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5806d-654f-4424-b028-485643be83f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:58.658843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5806d-654f-4424-b028-485643be83f5 admin admin] PATH_INFO: `/v3/projects/c404486ee19f4b5eb65586527cc470aa` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:58.661414 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5806d-654f-4424-b028-485643be83f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c404486ee19f4b5eb65586527cc470aa)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:58.665830 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5806d-654f-4424-b028-485643be83f5 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:58.735024 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bab6ed-949f-4bb0-abea-13eea4799dc9 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:58.752120 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8de5806d-654f-4424-b028-485643be83f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c404486ee19f4b5eb65586527cc470aa', 'request_id': 'req-8de5806d-654f-4424-b028-485643be83f5'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:58.760239 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8de5806d-654f-4424-b028-485643be83f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c404486ee19f4b5eb65586527cc470aa', 'request_id': 'req-8de5806d-654f-4424-b028-485643be83f5'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:58.773442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8de5806d-654f-4424-b028-485643be83f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c404486ee19f4b5eb65586527cc470aa', 'request_id': 'req-8de5806d-654f-4424-b028-485643be83f5'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:58.789770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8de5806d-654f-4424-b028-485643be83f5 admin admin] The token cache is being invalidate because project c404486ee19f4b5eb65586527cc470aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:58.790223 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8de5806d-654f-4424-b028-485643be83f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:58.791141 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8de5806d-654f-4424-b028-485643be83f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:58.793455 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 988/1974] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:58 2022] DELETE /identity/v3/projects/c404486ee19f4b5eb65586527cc470aa => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:45:58.799287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3d2ed-540d-4921-97e6-48e60b77de16 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:58.799748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3d2ed-540d-4921-97e6-48e60b77de16 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:58.809288 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3d2ed-540d-4921-97e6-48e60b77de16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:58.821128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0b1dd-e60f-4611-9e6b-b088933f0408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:58.821836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0b1dd-e60f-4611-9e6b-b088933f0408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:58.822143 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0b1dd-e60f-4611-9e6b-b088933f0408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:58.822410 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0b1dd-e60f-4611-9e6b-b088933f0408 admin admin] PATH_INFO: `/v3/users/29d386e4bfe64aeda348626dc3c315ac` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:58.823565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0b1dd-e60f-4611-9e6b-b088933f0408 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29d386e4bfe64aeda348626dc3c315ac)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:58.825615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0b1dd-e60f-4611-9e6b-b088933f0408 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:58.850326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:58.861260 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-7fc3d2ed-540d-4921-97e6-48e60b77de16 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:58.866068 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7fc3d2ed-540d-4921-97e6-48e60b77de16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:58.876228 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 989/1975] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:45:58.884255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32456d-8a6d-45ea-9132-1f93b5f3e7c3 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:58.884574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32456d-8a6d-45ea-9132-1f93b5f3e7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:58.884799 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32456d-8a6d-45ea-9132-1f93b5f3e7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:58.908021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-dd32456d-8a6d-45ea-9132-1f93b5f3e7c3 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:58.909305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-dd32456d-8a6d-45ea-9132-1f93b5f3e7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:58.921169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b9d0b1dd-e60f-4611-9e6b-b088933f0408 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29d386e4bfe64aeda348626dc3c315ac'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:58.930575 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 990/1976] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:45:58.936664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32456d-8a6d-45ea-9132-1f93b5f3e7c3 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:58.939379 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b9d0b1dd-e60f-4611-9e6b-b088933f0408 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29d386e4bfe64aeda348626dc3c315ac'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:58.940224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b9d0b1dd-e60f-4611-9e6b-b088933f0408 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29d386e4bfe64aeda348626dc3c315ac'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:58.940960 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b9d0b1dd-e60f-4611-9e6b-b088933f0408 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29d386e4bfe64aeda348626dc3c315ac'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:58.947071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32456d-8a6d-45ea-9132-1f93b5f3e7c3 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:58.952233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b81cf-1897-4e60-ad3e-9a72307ee265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:58.952886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b81cf-1897-4e60-ad3e-9a72307ee265 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:58.953204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b81cf-1897-4e60-ad3e-9a72307ee265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:58.953486 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b81cf-1897-4e60-ad3e-9a72307ee265 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:58.954154 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b81cf-1897-4e60-ad3e-9a72307ee265 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:45:58.954456 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b81cf-1897-4e60-ad3e-9a72307ee265 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:58.956081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b81cf-1897-4e60-ad3e-9a72307ee265 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:58.962486 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b9d0b1dd-e60f-4611-9e6b-b088933f0408 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29d386e4bfe64aeda348626dc3c315ac'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:58.973742 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 991/1977] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:45:58.978943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b81cf-1897-4e60-ad3e-9a72307ee265 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:58.981572 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-459b81cf-1897-4e60-ad3e-9a72307ee265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:58.983331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 987/1978] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:58 2022] DELETE /identity/v3/users/29d386e4bfe64aeda348626dc3c315ac => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:59.004534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b81cf-1897-4e60-ad3e-9a72307ee265 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:59.043512 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:59.100054 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c0a4a-9157-485c-8239-8d7669e91d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:59.100760 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c0a4a-9157-485c-8239-8d7669e91d00 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:59.101058 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c0a4a-9157-485c-8239-8d7669e91d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:59.101365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c0a4a-9157-485c-8239-8d7669e91d00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:59.102065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c0a4a-9157-485c-8239-8d7669e91d00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:59.104173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c0a4a-9157-485c-8239-8d7669e91d00 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:59.140039 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 992/1979] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:45:58 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:45:59.145383 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0b1dd-e60f-4611-9e6b-b088933f0408 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:59.158315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0b1dd-e60f-4611-9e6b-b088933f0408 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:59.239251 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb01bb6-4e1e-4aa9-b643-0e841f71bf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:59.239895 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb01bb6-4e1e-4aa9-b643-0e841f71bf3b admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:59.240202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb01bb6-4e1e-4aa9-b643-0e841f71bf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:59.240470 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb01bb6-4e1e-4aa9-b643-0e841f71bf3b admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:59.241124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb01bb6-4e1e-4aa9-b643-0e841f71bf3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:59.243171 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb01bb6-4e1e-4aa9-b643-0e841f71bf3b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:59.268035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 988/1980] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:45:59 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:45:59.272681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c0a4a-9157-485c-8239-8d7669e91d00 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:59.284340 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c0a4a-9157-485c-8239-8d7669e91d00 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:59.289945 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc08cdc-981f-460f-9e8e-92364f71b214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:59.290564 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc08cdc-981f-460f-9e8e-92364f71b214 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:59.290832 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc08cdc-981f-460f-9e8e-92364f71b214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:59.291121 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc08cdc-981f-460f-9e8e-92364f71b214 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:59.291775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc08cdc-981f-460f-9e8e-92364f71b214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:59.293512 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc08cdc-981f-460f-9e8e-92364f71b214 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:59.306827 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 993/1981] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:59.311248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb01bb6-4e1e-4aa9-b643-0e841f71bf3b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:59.326400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb01bb6-4e1e-4aa9-b643-0e841f71bf3b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:59.331589 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3b9a3-e780-44fd-8509-0f2f184eb45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:59.332295 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3b9a3-e780-44fd-8509-0f2f184eb45d admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:59.332575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3b9a3-e780-44fd-8509-0f2f184eb45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:59.332841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3b9a3-e780-44fd-8509-0f2f184eb45d admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:59.333577 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3b9a3-e780-44fd-8509-0f2f184eb45d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:59.335370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3b9a3-e780-44fd-8509-0f2f184eb45d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:59.344379 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc08cdc-981f-460f-9e8e-92364f71b214 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:59.346255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-bbc08cdc-981f-460f-9e8e-92364f71b214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:59.354596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 989/1982] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:45:59.356472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3b9a3-e780-44fd-8509-0f2f184eb45d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:59.360114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc08cdc-981f-460f-9e8e-92364f71b214 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:59.366210 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dddfe1a-5949-4c58-a639-8d3d7b290e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:59.371729 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddfe1a-5949-4c58-a639-8d3d7b290e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:59.372037 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3b9a3-e780-44fd-8509-0f2f184eb45d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:59.372196 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddfe1a-5949-4c58-a639-8d3d7b290e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:59.372369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddfe1a-5949-4c58-a639-8d3d7b290e53 admin admin] PATH_INFO: `/v3/projects/302f0d2ca4574f51b69a69a02f202823` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:59.373791 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dddfe1a-5949-4c58-a639-8d3d7b290e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=302f0d2ca4574f51b69a69a02f202823)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:59.375839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dddfe1a-5949-4c58-a639-8d3d7b290e53 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:59.378126 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490aae28-bd1b-4e1f-b044-00176947e098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:59.379021 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490aae28-bd1b-4e1f-b044-00176947e098 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:59.379210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490aae28-bd1b-4e1f-b044-00176947e098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:59.379419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490aae28-bd1b-4e1f-b044-00176947e098 admin admin] PATH_INFO: `/v3/projects/ee638507f9b0415f94f7c54e0e3db8cd/users/04c7152003c4491abea8305c26f366ed/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:59.400373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490aae28-bd1b-4e1f-b044-00176947e098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee638507f9b0415f94f7c54e0e3db8cd, user_id=04c7152003c4491abea8305c26f366ed, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:59.404421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490aae28-bd1b-4e1f-b044-00176947e098 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:59.425188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 990/1983] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:45:59 2022] PUT /identity/v3/projects/ee638507f9b0415f94f7c54e0e3db8cd/users/04c7152003c4491abea8305c26f366ed/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:45:59.432972 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454074f-3532-4ef0-8835-ddd7b6017251 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:59.433504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454074f-3532-4ef0-8835-ddd7b6017251 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:59.434002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454074f-3532-4ef0-8835-ddd7b6017251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:59.445030 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8dddfe1a-5949-4c58-a639-8d3d7b290e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '302f0d2ca4574f51b69a69a02f202823', 'request_id': 'req-8dddfe1a-5949-4c58-a639-8d3d7b290e53'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:59.452272 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8dddfe1a-5949-4c58-a639-8d3d7b290e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '302f0d2ca4574f51b69a69a02f202823', 'request_id': 'req-8dddfe1a-5949-4c58-a639-8d3d7b290e53'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:59.465521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8dddfe1a-5949-4c58-a639-8d3d7b290e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '302f0d2ca4574f51b69a69a02f202823', 'request_id': 'req-8dddfe1a-5949-4c58-a639-8d3d7b290e53'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:59.466564 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b454074f-3532-4ef0-8835-ddd7b6017251 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:45:59.468948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b454074f-3532-4ef0-8835-ddd7b6017251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:59.491111 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8dddfe1a-5949-4c58-a639-8d3d7b290e53 admin admin] The token cache is being invalidate because project 302f0d2ca4574f51b69a69a02f202823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:45:59.491613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8dddfe1a-5949-4c58-a639-8d3d7b290e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:59.492894 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8dddfe1a-5949-4c58-a639-8d3d7b290e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:59.496104 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 994/1984] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:45:59 2022] DELETE /identity/v3/projects/302f0d2ca4574f51b69a69a02f202823 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:45:59.503220 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dddfe1a-5949-4c58-a639-8d3d7b290e53 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:59.506843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-8dddfe1a-5949-4c58-a639-8d3d7b290e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:59.529781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:59.562509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 991/1985] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:45:59.576124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454074f-3532-4ef0-8835-ddd7b6017251 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:45:59.578363 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b454074f-3532-4ef0-8835-ddd7b6017251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:45:59.634513 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dddfe1a-5949-4c58-a639-8d3d7b290e53 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:59.654817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971be60-cb48-44c6-a71b-5e4b69cca92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:59.656718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971be60-cb48-44c6-a71b-5e4b69cca92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:59.656718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971be60-cb48-44c6-a71b-5e4b69cca92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:59.656718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971be60-cb48-44c6-a71b-5e4b69cca92e admin admin] PATH_INFO: `/v3/users/92cd6a97489542c7ba90b152b23c6656` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:59.657553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971be60-cb48-44c6-a71b-5e4b69cca92e admin admin] RBAC: Authorizing `identity:delete_user(user_id=92cd6a97489542c7ba90b152b23c6656)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:59.659572 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971be60-cb48-44c6-a71b-5e4b69cca92e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:59.687781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454074f-3532-4ef0-8835-ddd7b6017251 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:45:59.696869 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ff40e-40af-489a-8cc5-867412274fe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:45:59.697996 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ff40e-40af-489a-8cc5-867412274fe0 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:45:59.698507 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ff40e-40af-489a-8cc5-867412274fe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:45:59.699471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ff40e-40af-489a-8cc5-867412274fe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:45:59.700367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:45:59.722781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ff40e-40af-489a-8cc5-867412274fe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:45:59.724777 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ff40e-40af-489a-8cc5-867412274fe0 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:45:59.739765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c971be60-cb48-44c6-a71b-5e4b69cca92e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92cd6a97489542c7ba90b152b23c6656'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:59.743261 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 992/1986] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:45:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:45:59.751808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c971be60-cb48-44c6-a71b-5e4b69cca92e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92cd6a97489542c7ba90b152b23c6656'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:59.752936 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c971be60-cb48-44c6-a71b-5e4b69cca92e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92cd6a97489542c7ba90b152b23c6656'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:59.754227 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c971be60-cb48-44c6-a71b-5e4b69cca92e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92cd6a97489542c7ba90b152b23c6656'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:59.769179 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c971be60-cb48-44c6-a71b-5e4b69cca92e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92cd6a97489542c7ba90b152b23c6656'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:45:59.798648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 995/1987] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:45:59 2022] DELETE /identity/v3/users/92cd6a97489542c7ba90b152b23c6656 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:00.395984 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ff40e-40af-489a-8cc5-867412274fe0 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:00.413068 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ff40e-40af-489a-8cc5-867412274fe0 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:00.582824 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a1a16-2a14-47e9-b7dd-9828ce2c6a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:00.583491 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a1a16-2a14-47e9-b7dd-9828ce2c6a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:00.583777 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a1a16-2a14-47e9-b7dd-9828ce2c6a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:00.584056 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a1a16-2a14-47e9-b7dd-9828ce2c6a9a admin admin] PATH_INFO: `/v3/projects/1ba51b82aba94fce85ca5e4edaeaf1de` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:00.585412 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a1a16-2a14-47e9-b7dd-9828ce2c6a9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ba51b82aba94fce85ca5e4edaeaf1de)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:00.588288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a1a16-2a14-47e9-b7dd-9828ce2c6a9a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:00.663837 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7a5a1a16-2a14-47e9-b7dd-9828ce2c6a9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ba51b82aba94fce85ca5e4edaeaf1de', 'request_id': 'req-7a5a1a16-2a14-47e9-b7dd-9828ce2c6a9a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:00.674144 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7a5a1a16-2a14-47e9-b7dd-9828ce2c6a9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ba51b82aba94fce85ca5e4edaeaf1de', 'request_id': 'req-7a5a1a16-2a14-47e9-b7dd-9828ce2c6a9a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:00.688232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7a5a1a16-2a14-47e9-b7dd-9828ce2c6a9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ba51b82aba94fce85ca5e4edaeaf1de', 'request_id': 'req-7a5a1a16-2a14-47e9-b7dd-9828ce2c6a9a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:00.707539 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7a5a1a16-2a14-47e9-b7dd-9828ce2c6a9a admin admin] The token cache is being invalidate because project 1ba51b82aba94fce85ca5e4edaeaf1de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:00.708439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7a5a1a16-2a14-47e9-b7dd-9828ce2c6a9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:00.710036 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7a5a1a16-2a14-47e9-b7dd-9828ce2c6a9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:00.713493 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 993/1988] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:00 2022] DELETE /identity/v3/projects/1ba51b82aba94fce85ca5e4edaeaf1de => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:46:00.720439 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b5c27-07c4-4716-a40f-6e78a832c082 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:00.721685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b5c27-07c4-4716-a40f-6e78a832c082 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:00.722052 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b5c27-07c4-4716-a40f-6e78a832c082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:00.775643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-159b5c27-07c4-4716-a40f-6e78a832c082 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:00.778143 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-159b5c27-07c4-4716-a40f-6e78a832c082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:00.950655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7025782-9dfd-41db-b330-9a92693f14ff None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:00.951297 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7025782-9dfd-41db-b330-9a92693f14ff None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:00.951775 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7025782-9dfd-41db-b330-9a92693f14ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:00.957219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 996/1989] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:00.967774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300bf6d-f300-4c09-b567-fd89d59ad0dc None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:00.968287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300bf6d-f300-4c09-b567-fd89d59ad0dc None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:00.968775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300bf6d-f300-4c09-b567-fd89d59ad0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:01.013613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-1300bf6d-f300-4c09-b567-fd89d59ad0dc None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:01.016286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-1300bf6d-f300-4c09-b567-fd89d59ad0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:01.031135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 997/1990] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:01.038628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300bf6d-f300-4c09-b567-fd89d59ad0dc None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:01.054239 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f7025782-9dfd-41db-b330-9a92693f14ff None None] MFA Rules not processed for user `04c7152003c4491abea8305c26f366ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:01.057639 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f7025782-9dfd-41db-b330-9a92693f14ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:01.067288 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300bf6d-f300-4c09-b567-fd89d59ad0dc None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:01.078004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e2284-1fad-4fb9-8f69-f548990b8b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:01.079619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2284-1fad-4fb9-8f69-f548990b8b35 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:01.080635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2284-1fad-4fb9-8f69-f548990b8b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:01.081553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e2284-1fad-4fb9-8f69-f548990b8b35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:01.083321 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e2284-1fad-4fb9-8f69-f548990b8b35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:46:01.084233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e2284-1fad-4fb9-8f69-f548990b8b35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:01.088691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e2284-1fad-4fb9-8f69-f548990b8b35 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:01.106254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:01.129061 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 998/1991] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:46:01.134768 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e2284-1fad-4fb9-8f69-f548990b8b35 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:01.163057 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e2284-1fad-4fb9-8f69-f548990b8b35 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:01.188618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd69ebc-430a-4356-81fd-335b5c851ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:01.193047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd69ebc-430a-4356-81fd-335b5c851ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:01.193563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd69ebc-430a-4356-81fd-335b5c851ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:01.194263 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd69ebc-430a-4356-81fd-335b5c851ce1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:01.195333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd69ebc-430a-4356-81fd-335b5c851ce1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:01.203603 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd69ebc-430a-4356-81fd-335b5c851ce1 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:01.298898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 994/1992] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:01.315981 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7025782-9dfd-41db-b330-9a92693f14ff None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:01.319167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f7025782-9dfd-41db-b330-9a92693f14ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:01.325884 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 999/1993] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:46:01 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:01.328494 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd69ebc-430a-4356-81fd-335b5c851ce1 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:01.437872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7025782-9dfd-41db-b330-9a92693f14ff None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:01.441610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd69ebc-430a-4356-81fd-335b5c851ce1 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:01.449642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4915fa-3677-4bfd-b1d1-154a6eac3443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:01.450697 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4915fa-3677-4bfd-b1d1-154a6eac3443 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:01.451150 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4915fa-3677-4bfd-b1d1-154a6eac3443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:01.451656 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4915fa-3677-4bfd-b1d1-154a6eac3443 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:01.452688 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4915fa-3677-4bfd-b1d1-154a6eac3443 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:01.456303 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4915fa-3677-4bfd-b1d1-154a6eac3443 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:01.470935 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:01.492643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1000/1994] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:46:01 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:01.498663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4915fa-3677-4bfd-b1d1-154a6eac3443 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:01.512639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4915fa-3677-4bfd-b1d1-154a6eac3443 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:01.519311 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a010f4-3cf8-486a-a872-4c2d960a35be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:01.520284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a010f4-3cf8-486a-a872-4c2d960a35be admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:01.520703 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a010f4-3cf8-486a-a872-4c2d960a35be admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:01.521182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a010f4-3cf8-486a-a872-4c2d960a35be admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:01.522046 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a010f4-3cf8-486a-a872-4c2d960a35be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:01.523959 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a010f4-3cf8-486a-a872-4c2d960a35be admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:01.530945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdd77a-d3e0-4246-94d7-4022e9cb21be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:01.532065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdd77a-d3e0-4246-94d7-4022e9cb21be service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:01.532487 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdd77a-d3e0-4246-94d7-4022e9cb21be service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:01.533013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdd77a-d3e0-4246-94d7-4022e9cb21be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:01.537493 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1001/1995] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:46:01.541544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a010f4-3cf8-486a-a872-4c2d960a35be admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:01.552416 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a010f4-3cf8-486a-a872-4c2d960a35be admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:01.557849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b1c5e-8e75-46f2-870c-44c8cb65263d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:01.558481 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b1c5e-8e75-46f2-870c-44c8cb65263d admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:01.558789 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b1c5e-8e75-46f2-870c-44c8cb65263d admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:01.559036 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b1c5e-8e75-46f2-870c-44c8cb65263d admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:01.559705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b1c5e-8e75-46f2-870c-44c8cb65263d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:01.561402 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b1c5e-8e75-46f2-870c-44c8cb65263d admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:01.573666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1002/1996] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:46:01.578001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b1c5e-8e75-46f2-870c-44c8cb65263d admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:01.589858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b1c5e-8e75-46f2-870c-44c8cb65263d admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:01.595877 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec526f1-59e0-48f0-8825-154df38768ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:01.596732 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec526f1-59e0-48f0-8825-154df38768ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:01.597256 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec526f1-59e0-48f0-8825-154df38768ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:01.597713 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec526f1-59e0-48f0-8825-154df38768ee admin admin] PATH_INFO: `/v3/projects/d29ff77d79f64e7cb98b5ef3ee37db6c/users/1cc790daaece42a4afed6b51d53d9ec7/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:01.616137 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec526f1-59e0-48f0-8825-154df38768ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d29ff77d79f64e7cb98b5ef3ee37db6c, user_id=1cc790daaece42a4afed6b51d53d9ec7, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:01.618923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec526f1-59e0-48f0-8825-154df38768ee admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:01.636527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1003/1997] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:01 2022] PUT /identity/v3/projects/d29ff77d79f64e7cb98b5ef3ee37db6c/users/1cc790daaece42a4afed6b51d53d9ec7/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:46:01.648644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfdd77a-d3e0-4246-94d7-4022e9cb21be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:01.648874 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11737e-f7bf-4d24-a0a4-c44f3f92bb45 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:01.649058 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11737e-f7bf-4d24-a0a4-c44f3f92bb45 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:01.649527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11737e-f7bf-4d24-a0a4-c44f3f92bb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:01.650922 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfdd77a-d3e0-4246-94d7-4022e9cb21be service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:01.662466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 995/1998] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:46:01.699496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-3f11737e-f7bf-4d24-a0a4-c44f3f92bb45 None None] MFA Rules not processed for user `1cc790daaece42a4afed6b51d53d9ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:01.701860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-3f11737e-f7bf-4d24-a0a4-c44f3f92bb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:01.794142 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1004/1999] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:01.810816 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdd77a-d3e0-4246-94d7-4022e9cb21be service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:01.812685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4dfdd77a-d3e0-4246-94d7-4022e9cb21be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:01.828514 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdd77a-d3e0-4246-94d7-4022e9cb21be service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:01.860770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:01.916575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc9c0a-7c2c-4a8a-aac4-fd6bdb845f4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:01.917505 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc9c0a-7c2c-4a8a-aac4-fd6bdb845f4b service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:01.917800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc9c0a-7c2c-4a8a-aac4-fd6bdb845f4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:01.918136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc9c0a-7c2c-4a8a-aac4-fd6bdb845f4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:01.928116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc9c0a-7c2c-4a8a-aac4-fd6bdb845f4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:01.929632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc9c0a-7c2c-4a8a-aac4-fd6bdb845f4b service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:01.940439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 996/2000] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:02.223353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11737e-f7bf-4d24-a0a4-c44f3f92bb45 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:02.225210 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-3f11737e-f7bf-4d24-a0a4-c44f3f92bb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:02.365587 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11737e-f7bf-4d24-a0a4-c44f3f92bb45 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:02.375254 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539390e-bcfe-43eb-8e25-c48916d609c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:02.376614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539390e-bcfe-43eb-8e25-c48916d609c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:02.377322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539390e-bcfe-43eb-8e25-c48916d609c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:02.377907 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539390e-bcfe-43eb-8e25-c48916d609c6 admin admin] PATH_INFO: `/v3/users/9871b09c5b424746af3c19bcde6a0670` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:02.381077 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539390e-bcfe-43eb-8e25-c48916d609c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9871b09c5b424746af3c19bcde6a0670)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:02.385905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539390e-bcfe-43eb-8e25-c48916d609c6 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:02.489193 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:02.504588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5539390e-bcfe-43eb-8e25-c48916d609c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9871b09c5b424746af3c19bcde6a0670'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:02.541670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5539390e-bcfe-43eb-8e25-c48916d609c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9871b09c5b424746af3c19bcde6a0670'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:02.542928 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5539390e-bcfe-43eb-8e25-c48916d609c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9871b09c5b424746af3c19bcde6a0670'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:02.544010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5539390e-bcfe-43eb-8e25-c48916d609c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9871b09c5b424746af3c19bcde6a0670'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:02.555421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5539390e-bcfe-43eb-8e25-c48916d609c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9871b09c5b424746af3c19bcde6a0670'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:02.587424 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1005/2001] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:02 2022] DELETE /identity/v3/users/9871b09c5b424746af3c19bcde6a0670 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:46:02.589407 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131352b6-eeba-4b32-8f17-4085bf380f9d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:02.589906 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131352b6-eeba-4b32-8f17-4085bf380f9d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:02.590206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131352b6-eeba-4b32-8f17-4085bf380f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:02.611964 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-131352b6-eeba-4b32-8f17-4085bf380f9d None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:02.613339 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-131352b6-eeba-4b32-8f17-4085bf380f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:02.687272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 997/2002] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:02.695170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539390e-bcfe-43eb-8e25-c48916d609c6 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:02.696927 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5539390e-bcfe-43eb-8e25-c48916d609c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:02.772710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539390e-bcfe-43eb-8e25-c48916d609c6 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:02.777746 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd60b3-2e80-4fd8-858b-1e9081004d3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:02.778520 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd60b3-2e80-4fd8-858b-1e9081004d3b service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:02.778880 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd60b3-2e80-4fd8-858b-1e9081004d3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:02.779272 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd60b3-2e80-4fd8-858b-1e9081004d3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:02.791148 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd60b3-2e80-4fd8-858b-1e9081004d3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:02.792665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd60b3-2e80-4fd8-858b-1e9081004d3b service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:02.804453 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1006/2003] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:03.172142 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131352b6-eeba-4b32-8f17-4085bf380f9d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:03.174400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-131352b6-eeba-4b32-8f17-4085bf380f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:03.195297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045408e-535c-47b4-9094-2ecba87399fb None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:03.197487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045408e-535c-47b4-9094-2ecba87399fb None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:03.197487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045408e-535c-47b4-9094-2ecba87399fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:03.200384 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131352b6-eeba-4b32-8f17-4085bf380f9d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:03.208483 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b4b34-e908-4a2f-8dc4-0547854f1504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:03.209587 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b4b34-e908-4a2f-8dc4-0547854f1504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:03.209865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b4b34-e908-4a2f-8dc4-0547854f1504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:03.210129 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b4b34-e908-4a2f-8dc4-0547854f1504 admin admin] PATH_INFO: `/v3/projects/2251a89cc2a74e55895b6b9b1618b802` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:03.211791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b4b34-e908-4a2f-8dc4-0547854f1504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2251a89cc2a74e55895b6b9b1618b802)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:03.213945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b4b34-e908-4a2f-8dc4-0547854f1504 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:03.232646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-7045408e-535c-47b4-9094-2ecba87399fb None None] MFA Rules not processed for user `04c7152003c4491abea8305c26f366ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:03.235429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7045408e-535c-47b4-9094-2ecba87399fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:03.296933 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5e1b4b34-e908-4a2f-8dc4-0547854f1504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2251a89cc2a74e55895b6b9b1618b802', 'request_id': 'req-5e1b4b34-e908-4a2f-8dc4-0547854f1504'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:03.305649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5e1b4b34-e908-4a2f-8dc4-0547854f1504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2251a89cc2a74e55895b6b9b1618b802', 'request_id': 'req-5e1b4b34-e908-4a2f-8dc4-0547854f1504'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:03.317291 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5e1b4b34-e908-4a2f-8dc4-0547854f1504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2251a89cc2a74e55895b6b9b1618b802', 'request_id': 'req-5e1b4b34-e908-4a2f-8dc4-0547854f1504'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:03.324787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1007/2004] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:03.333816 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5e1b4b34-e908-4a2f-8dc4-0547854f1504 admin admin] The token cache is being invalidate because project 2251a89cc2a74e55895b6b9b1618b802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:03.334340 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5e1b4b34-e908-4a2f-8dc4-0547854f1504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:03.335179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5e1b4b34-e908-4a2f-8dc4-0547854f1504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:03.337772 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 998/2005] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:03 2022] DELETE /identity/v3/projects/2251a89cc2a74e55895b6b9b1618b802 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:03.344097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045408e-535c-47b4-9094-2ecba87399fb None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:03.364031 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045408e-535c-47b4-9094-2ecba87399fb None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:03.457908 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ba57f-29e6-49d4-9eb9-261c5366b174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:03.457908 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ba57f-29e6-49d4-9eb9-261c5366b174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:03.458550 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ba57f-29e6-49d4-9eb9-261c5366b174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:03.459003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ba57f-29e6-49d4-9eb9-261c5366b174 admin admin] PATH_INFO: `/v3/users/bdd5d3d4eb154acca5eb65f4a1048308` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:03.460553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ba57f-29e6-49d4-9eb9-261c5366b174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdd5d3d4eb154acca5eb65f4a1048308)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:03.463341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ba57f-29e6-49d4-9eb9-261c5366b174 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:03.466588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:03.548129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-240ba57f-29e6-49d4-9eb9-261c5366b174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdd5d3d4eb154acca5eb65f4a1048308'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:03.573526 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-240ba57f-29e6-49d4-9eb9-261c5366b174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdd5d3d4eb154acca5eb65f4a1048308'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:03.574268 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-240ba57f-29e6-49d4-9eb9-261c5366b174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdd5d3d4eb154acca5eb65f4a1048308'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:03.575288 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-240ba57f-29e6-49d4-9eb9-261c5366b174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdd5d3d4eb154acca5eb65f4a1048308'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:03.587104 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-240ba57f-29e6-49d4-9eb9-261c5366b174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdd5d3d4eb154acca5eb65f4a1048308'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:03.610081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1008/2006] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:03 2022] DELETE /identity/v3/users/bdd5d3d4eb154acca5eb65f4a1048308 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:04.248307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b4b34-e908-4a2f-8dc4-0547854f1504 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:04.251108 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5e1b4b34-e908-4a2f-8dc4-0547854f1504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:04.273560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b4b34-e908-4a2f-8dc4-0547854f1504 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:04.346925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:04.368450 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:04.369303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:04.369676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:04.370043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf admin admin] PATH_INFO: `/v3/projects/fc7f5fbf7a2d491f9550117c2f3f4f73` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:04.371588 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc7f5fbf7a2d491f9550117c2f3f4f73)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:04.373983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:04.450561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc7f5fbf7a2d491f9550117c2f3f4f73', 'request_id': 'req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:04.461152 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc7f5fbf7a2d491f9550117c2f3f4f73', 'request_id': 'req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:04.475367 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc7f5fbf7a2d491f9550117c2f3f4f73', 'request_id': 'req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:04.492254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf admin admin] The token cache is being invalidate because project fc7f5fbf7a2d491f9550117c2f3f4f73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:04.492742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:04.493856 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:04.496156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 999/2007] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:04 2022] DELETE /identity/v3/projects/fc7f5fbf7a2d491f9550117c2f3f4f73 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:04.500443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ba57f-29e6-49d4-9eb9-261c5366b174 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:04.502409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-240ba57f-29e6-49d4-9eb9-261c5366b174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:04.520317 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ba57f-29e6-49d4-9eb9-261c5366b174 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:04.640456 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fd6a6-c1ae-4061-ad7e-c8bf8a3c54de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:04.641291 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fd6a6-c1ae-4061-ad7e-c8bf8a3c54de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:04.641744 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fd6a6-c1ae-4061-ad7e-c8bf8a3c54de admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:04.642103 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fd6a6-c1ae-4061-ad7e-c8bf8a3c54de admin admin] PATH_INFO: `/v3/users/79820bbdeae24db69d975f0d3468cf5b` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:04.643860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fd6a6-c1ae-4061-ad7e-c8bf8a3c54de admin admin] RBAC: Authorizing `identity:delete_user(user_id=79820bbdeae24db69d975f0d3468cf5b)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:04.646822 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fd6a6-c1ae-4061-ad7e-c8bf8a3c54de admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:04.723168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-274fd6a6-c1ae-4061-ad7e-c8bf8a3c54de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79820bbdeae24db69d975f0d3468cf5b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:04.737010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-274fd6a6-c1ae-4061-ad7e-c8bf8a3c54de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79820bbdeae24db69d975f0d3468cf5b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:04.738720 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-274fd6a6-c1ae-4061-ad7e-c8bf8a3c54de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79820bbdeae24db69d975f0d3468cf5b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:04.741378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-274fd6a6-c1ae-4061-ad7e-c8bf8a3c54de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79820bbdeae24db69d975f0d3468cf5b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:04.755979 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-274fd6a6-c1ae-4061-ad7e-c8bf8a3c54de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79820bbdeae24db69d975f0d3468cf5b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:04.790058 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:04.802613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1009/2008] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:04 2022] DELETE /identity/v3/users/79820bbdeae24db69d975f0d3468cf5b => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:05.218470 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:05.220833 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:05.258718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2cf6d-211f-4e1a-84b6-44ab5fcc7adf admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:05.392027 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7ae30-b468-4306-ace4-c09f29e2e574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:05.392724 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7ae30-b468-4306-ace4-c09f29e2e574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:05.393516 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7ae30-b468-4306-ace4-c09f29e2e574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:05.393820 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7ae30-b468-4306-ace4-c09f29e2e574 admin admin] PATH_INFO: `/v3/projects/10a87b1feaea474ab429df8fee9a62a0` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:05.395227 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f7ae30-b468-4306-ace4-c09f29e2e574 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10a87b1feaea474ab429df8fee9a62a0)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:05.397234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f7ae30-b468-4306-ace4-c09f29e2e574 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:05.464611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-02f7ae30-b468-4306-ace4-c09f29e2e574 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10a87b1feaea474ab429df8fee9a62a0', 'request_id': 'req-02f7ae30-b468-4306-ace4-c09f29e2e574'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:05.472124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-02f7ae30-b468-4306-ace4-c09f29e2e574 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10a87b1feaea474ab429df8fee9a62a0', 'request_id': 'req-02f7ae30-b468-4306-ace4-c09f29e2e574'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:05.490746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-02f7ae30-b468-4306-ace4-c09f29e2e574 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10a87b1feaea474ab429df8fee9a62a0', 'request_id': 'req-02f7ae30-b468-4306-ace4-c09f29e2e574'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:05.511989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-02f7ae30-b468-4306-ace4-c09f29e2e574 admin admin] The token cache is being invalidate because project 10a87b1feaea474ab429df8fee9a62a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:05.513105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-02f7ae30-b468-4306-ace4-c09f29e2e574 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:05.514080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-02f7ae30-b468-4306-ace4-c09f29e2e574 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:05.520628 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1000/2009] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:05 2022] DELETE /identity/v3/projects/10a87b1feaea474ab429df8fee9a62a0 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:05.523342 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e0dc9-623e-4d3f-9c8b-d1f3e6be2f87 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:05.523876 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e0dc9-623e-4d3f-9c8b-d1f3e6be2f87 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:05.524587 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e0dc9-623e-4d3f-9c8b-d1f3e6be2f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:05.553257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-e75e0dc9-623e-4d3f-9c8b-d1f3e6be2f87 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:05.554950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-e75e0dc9-623e-4d3f-9c8b-d1f3e6be2f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:05.647980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1010/2010] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:05.656695 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6f664-4c9d-4922-beef-4468ae4aca2e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:05.656968 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6f664-4c9d-4922-beef-4468ae4aca2e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:05.657355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6f664-4c9d-4922-beef-4468ae4aca2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:05.672262 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:05.678600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d1a6f664-4c9d-4922-beef-4468ae4aca2e None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:05.680724 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d1a6f664-4c9d-4922-beef-4468ae4aca2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:05.690111 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1001/2011] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:05.695448 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e0dc9-623e-4d3f-9c8b-d1f3e6be2f87 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:05.712942 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e0dc9-623e-4d3f-9c8b-d1f3e6be2f87 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:05.719897 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012298df-6ddb-4677-a514-1022f28e948b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:05.720857 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012298df-6ddb-4677-a514-1022f28e948b admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:05.721306 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012298df-6ddb-4677-a514-1022f28e948b admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:05.721675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012298df-6ddb-4677-a514-1022f28e948b admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:05.722628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012298df-6ddb-4677-a514-1022f28e948b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:46:05.723073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012298df-6ddb-4677-a514-1022f28e948b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:05.724992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012298df-6ddb-4677-a514-1022f28e948b admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:05.742518 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1011/2012] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:46:05.748224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6f664-4c9d-4922-beef-4468ae4aca2e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:05.772139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6f664-4c9d-4922-beef-4468ae4aca2e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:05.782235 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35544b3c-141e-4ab5-8a7f-92ac5032883e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:05.783400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35544b3c-141e-4ab5-8a7f-92ac5032883e admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:05.783969 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35544b3c-141e-4ab5-8a7f-92ac5032883e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:05.784424 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35544b3c-141e-4ab5-8a7f-92ac5032883e admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:05.785958 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35544b3c-141e-4ab5-8a7f-92ac5032883e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:05.790237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35544b3c-141e-4ab5-8a7f-92ac5032883e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:05.845879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1002/2013] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:46:05 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:05.847842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012298df-6ddb-4677-a514-1022f28e948b admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:05.867384 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012298df-6ddb-4677-a514-1022f28e948b admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:05.962424 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb7d30-9a41-4a87-a25f-dc2ffca13b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:05.962980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb7d30-9a41-4a87-a25f-dc2ffca13b37 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:05.963252 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb7d30-9a41-4a87-a25f-dc2ffca13b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:05.963531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb7d30-9a41-4a87-a25f-dc2ffca13b37 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:05.964179 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb7d30-9a41-4a87-a25f-dc2ffca13b37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:05.966224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb7d30-9a41-4a87-a25f-dc2ffca13b37 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:06.002788 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1012/2014] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:46:05 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:06.009092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35544b3c-141e-4ab5-8a7f-92ac5032883e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:06.024628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb7d30-9a41-4a87-a25f-dc2ffca13b37 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:06.027440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-82cb7d30-9a41-4a87-a25f-dc2ffca13b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:06.061167 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb7d30-9a41-4a87-a25f-dc2ffca13b37 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:06.070917 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35544b3c-141e-4ab5-8a7f-92ac5032883e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:06.071113 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3f770-adfd-4bb9-bea2-83d4643f3ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:06.071906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3f770-adfd-4bb9-bea2-83d4643f3ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:06.072459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3f770-adfd-4bb9-bea2-83d4643f3ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:06.073011 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3f770-adfd-4bb9-bea2-83d4643f3ab7 admin admin] PATH_INFO: `/v3/users/1cc790daaece42a4afed6b51d53d9ec7` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:06.075712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3f770-adfd-4bb9-bea2-83d4643f3ab7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cc790daaece42a4afed6b51d53d9ec7)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:06.084343 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3f770-adfd-4bb9-bea2-83d4643f3ab7 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:06.091816 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50a744-30db-4a83-9ff4-210a716b6724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:06.093009 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50a744-30db-4a83-9ff4-210a716b6724 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:06.093637 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50a744-30db-4a83-9ff4-210a716b6724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:06.093985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50a744-30db-4a83-9ff4-210a716b6724 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:06.095014 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50a744-30db-4a83-9ff4-210a716b6724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:06.096929 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50a744-30db-4a83-9ff4-210a716b6724 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:06.116772 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1003/2015] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:06.118201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50a744-30db-4a83-9ff4-210a716b6724 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:06.131479 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50a744-30db-4a83-9ff4-210a716b6724 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:06.142371 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ef627-f0e5-4e0a-8135-9289f7b7c8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:06.143039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ef627-f0e5-4e0a-8135-9289f7b7c8fb admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:06.143325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ef627-f0e5-4e0a-8135-9289f7b7c8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:06.143596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ef627-f0e5-4e0a-8135-9289f7b7c8fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:06.144623 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ef627-f0e5-4e0a-8135-9289f7b7c8fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:06.150154 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ef627-f0e5-4e0a-8135-9289f7b7c8fb admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:06.179600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1004/2016] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:46:06.184652 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ef627-f0e5-4e0a-8135-9289f7b7c8fb admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:06.189168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-89b3f770-adfd-4bb9-bea2-83d4643f3ab7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cc790daaece42a4afed6b51d53d9ec7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:06.219163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ef627-f0e5-4e0a-8135-9289f7b7c8fb admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:06.219451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-89b3f770-adfd-4bb9-bea2-83d4643f3ab7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cc790daaece42a4afed6b51d53d9ec7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:06.219451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-89b3f770-adfd-4bb9-bea2-83d4643f3ab7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cc790daaece42a4afed6b51d53d9ec7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:06.219451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-89b3f770-adfd-4bb9-bea2-83d4643f3ab7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cc790daaece42a4afed6b51d53d9ec7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:06.235075 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-89b3f770-adfd-4bb9-bea2-83d4643f3ab7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cc790daaece42a4afed6b51d53d9ec7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:06.239980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320a0c3-d6a2-42cb-8fbb-76b905f50129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:06.239980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320a0c3-d6a2-42cb-8fbb-76b905f50129 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:06.239980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320a0c3-d6a2-42cb-8fbb-76b905f50129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:06.239980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320a0c3-d6a2-42cb-8fbb-76b905f50129 admin admin] PATH_INFO: `/v3/projects/fc2db39918144652925651a08d3bb2cf/users/723f924b01a148efbeab1051015890a4/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:06.254540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:06.274374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320a0c3-d6a2-42cb-8fbb-76b905f50129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc2db39918144652925651a08d3bb2cf, user_id=723f924b01a148efbeab1051015890a4, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:06.280008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1013/2017] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:06 2022] DELETE /identity/v3/users/1cc790daaece42a4afed6b51d53d9ec7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:06.281639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef67e5b-a60c-48cd-8f03-5f835556fefe None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:06.281928 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef67e5b-a60c-48cd-8f03-5f835556fefe None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:06.282205 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef67e5b-a60c-48cd-8f03-5f835556fefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:06.282642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320a0c3-d6a2-42cb-8fbb-76b905f50129 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:06.309011 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-4ef67e5b-a60c-48cd-8f03-5f835556fefe None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:06.310603 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4ef67e5b-a60c-48cd-8f03-5f835556fefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:06.313831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1005/2018] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:06 2022] PUT /identity/v3/projects/fc2db39918144652925651a08d3bb2cf/users/723f924b01a148efbeab1051015890a4/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:06.333761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6f1fe-b46b-49c3-a26a-ae9168e233ec None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:06.334161 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6f1fe-b46b-49c3-a26a-ae9168e233ec None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:06.334665 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6f1fe-b46b-49c3-a26a-ae9168e233ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:06.386552 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f4b6f1fe-b46b-49c3-a26a-ae9168e233ec None None] MFA Rules not processed for user `723f924b01a148efbeab1051015890a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:06.389273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f4b6f1fe-b46b-49c3-a26a-ae9168e233ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:06.424575 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1014/2019] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:06.435009 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef67e5b-a60c-48cd-8f03-5f835556fefe None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:06.436887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4ef67e5b-a60c-48cd-8f03-5f835556fefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:06.506009 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1006/2020] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:06.511181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6f1fe-b46b-49c3-a26a-ae9168e233ec None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:06.513038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f4b6f1fe-b46b-49c3-a26a-ae9168e233ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:06.530211 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef67e5b-a60c-48cd-8f03-5f835556fefe None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:06.531308 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6f1fe-b46b-49c3-a26a-ae9168e233ec None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:06.536172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09029ade-6b1c-4af2-a350-bebfc5676df6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:06.537191 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09029ade-6b1c-4af2-a350-bebfc5676df6 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:06.537641 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09029ade-6b1c-4af2-a350-bebfc5676df6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:06.538082 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09029ade-6b1c-4af2-a350-bebfc5676df6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:06.540073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45146a8-bed4-49da-896a-9b764571a053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:06.541343 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45146a8-bed4-49da-896a-9b764571a053 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:06.541988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45146a8-bed4-49da-896a-9b764571a053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:06.542893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45146a8-bed4-49da-896a-9b764571a053 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:06.544805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45146a8-bed4-49da-896a-9b764571a053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:06.549081 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45146a8-bed4-49da-896a-9b764571a053 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:06.555031 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09029ade-6b1c-4af2-a350-bebfc5676df6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:06.559075 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09029ade-6b1c-4af2-a350-bebfc5676df6 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:06.579515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1015/2021] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:46:06.601277 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1007/2022] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:46:06 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:46:06.604273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09029ade-6b1c-4af2-a350-bebfc5676df6 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:06.616454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09029ade-6b1c-4af2-a350-bebfc5676df6 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:06.697184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb615f-3d53-4557-bbda-1ff5b365abdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:06.697184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb615f-3d53-4557-bbda-1ff5b365abdd admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:06.697184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb615f-3d53-4557-bbda-1ff5b365abdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:06.697184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb615f-3d53-4557-bbda-1ff5b365abdd admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:06.697898 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb615f-3d53-4557-bbda-1ff5b365abdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:06.709376 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb615f-3d53-4557-bbda-1ff5b365abdd admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:06.749866 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1016/2023] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:46:06 2022] POST /identity/v3/users => generated 295 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:06.761186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45146a8-bed4-49da-896a-9b764571a053 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:06.772115 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45146a8-bed4-49da-896a-9b764571a053 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:06.778856 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb58d58-d096-4618-a083-e9494c155816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:06.779511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb58d58-d096-4618-a083-e9494c155816 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:06.779772 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb58d58-d096-4618-a083-e9494c155816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:06.780062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb58d58-d096-4618-a083-e9494c155816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:06.780691 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb58d58-d096-4618-a083-e9494c155816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:06.782843 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb58d58-d096-4618-a083-e9494c155816 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:06.794916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1008/2024] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:06.799211 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb615f-3d53-4557-bbda-1ff5b365abdd admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:06.811519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb615f-3d53-4557-bbda-1ff5b365abdd admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:06.818967 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63ec99-1a19-4991-b137-68e0d55f0422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:06.820117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63ec99-1a19-4991-b137-68e0d55f0422 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:06.820472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63ec99-1a19-4991-b137-68e0d55f0422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:06.820820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63ec99-1a19-4991-b137-68e0d55f0422 admin admin] PATH_INFO: `/v3/projects/2ea435e53389452bb323644295c1b296/users/1a7c89730c034d3eb58a7cf6cac2bd56/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:06.849991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63ec99-1a19-4991-b137-68e0d55f0422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ea435e53389452bb323644295c1b296, user_id=1a7c89730c034d3eb58a7cf6cac2bd56, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:06.855589 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63ec99-1a19-4991-b137-68e0d55f0422 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:06.869168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:06.877513 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1017/2025] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:06 2022] PUT /identity/v3/projects/2ea435e53389452bb323644295c1b296/users/1a7c89730c034d3eb58a7cf6cac2bd56/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:06.882923 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb58d58-d096-4618-a083-e9494c155816 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:06.898503 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb58d58-d096-4618-a083-e9494c155816 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:06.983316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:07.006504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ee404-ca8b-48ff-9b9a-822ce5336e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:07.007193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ee404-ca8b-48ff-9b9a-822ce5336e6b admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:07.007650 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ee404-ca8b-48ff-9b9a-822ce5336e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:07.007931 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ee404-ca8b-48ff-9b9a-822ce5336e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:07.008669 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ee404-ca8b-48ff-9b9a-822ce5336e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:07.010607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ee404-ca8b-48ff-9b9a-822ce5336e6b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:07.023697 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1009/2026] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:07.032122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63ec99-1a19-4991-b137-68e0d55f0422 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:07.039471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63ec99-1a19-4991-b137-68e0d55f0422 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:07.044765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48376b2b-0ccd-4cdf-a9cc-33f02f3eb4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:07.045498 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48376b2b-0ccd-4cdf-a9cc-33f02f3eb4ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:07.046049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48376b2b-0ccd-4cdf-a9cc-33f02f3eb4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:07.046049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48376b2b-0ccd-4cdf-a9cc-33f02f3eb4ac admin admin] PATH_INFO: `/v3/projects/2ea435e53389452bb323644295c1b296/users/1a7c89730c034d3eb58a7cf6cac2bd56/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:07.048522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48376b2b-0ccd-4cdf-a9cc-33f02f3eb4ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ea435e53389452bb323644295c1b296, user_id=1a7c89730c034d3eb58a7cf6cac2bd56, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:07.050713 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48376b2b-0ccd-4cdf-a9cc-33f02f3eb4ac admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:07.063338 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1018/2027] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:07 2022] PUT /identity/v3/projects/2ea435e53389452bb323644295c1b296/users/1a7c89730c034d3eb58a7cf6cac2bd56/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:46:07.075858 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af4a8c-a11f-4748-84d2-61eadbfdbf1c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:07.076309 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af4a8c-a11f-4748-84d2-61eadbfdbf1c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:07.076684 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af4a8c-a11f-4748-84d2-61eadbfdbf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:07.092871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48376b2b-0ccd-4cdf-a9cc-33f02f3eb4ac admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:07.096108 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-48376b2b-0ccd-4cdf-a9cc-33f02f3eb4ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:07.112216 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f8af4a8c-a11f-4748-84d2-61eadbfdbf1c None None] MFA Rules not processed for user `1a7c89730c034d3eb58a7cf6cac2bd56`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:07.113706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f8af4a8c-a11f-4748-84d2-61eadbfdbf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:07.182041 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48376b2b-0ccd-4cdf-a9cc-33f02f3eb4ac admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:07.187522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1206d6e-7a67-4804-80fd-37784b815d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:07.188260 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1206d6e-7a67-4804-80fd-37784b815d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:07.188550 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1206d6e-7a67-4804-80fd-37784b815d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:07.188818 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1206d6e-7a67-4804-80fd-37784b815d4e admin admin] PATH_INFO: `/v3/projects/d29ff77d79f64e7cb98b5ef3ee37db6c` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:07.190076 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1206d6e-7a67-4804-80fd-37784b815d4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d29ff77d79f64e7cb98b5ef3ee37db6c)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:07.193448 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1206d6e-7a67-4804-80fd-37784b815d4e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:07.202349 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1010/2028] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:07.215817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af4a8c-a11f-4748-84d2-61eadbfdbf1c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:07.218371 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f8af4a8c-a11f-4748-84d2-61eadbfdbf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:07.276529 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c1206d6e-7a67-4804-80fd-37784b815d4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd29ff77d79f64e7cb98b5ef3ee37db6c', 'request_id': 'req-c1206d6e-7a67-4804-80fd-37784b815d4e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:07.285216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c1206d6e-7a67-4804-80fd-37784b815d4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd29ff77d79f64e7cb98b5ef3ee37db6c', 'request_id': 'req-c1206d6e-7a67-4804-80fd-37784b815d4e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:07.299589 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c1206d6e-7a67-4804-80fd-37784b815d4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd29ff77d79f64e7cb98b5ef3ee37db6c', 'request_id': 'req-c1206d6e-7a67-4804-80fd-37784b815d4e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:07.318245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c1206d6e-7a67-4804-80fd-37784b815d4e admin admin] The token cache is being invalidate because project d29ff77d79f64e7cb98b5ef3ee37db6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:07.318424 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af4a8c-a11f-4748-84d2-61eadbfdbf1c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:07.318875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c1206d6e-7a67-4804-80fd-37784b815d4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:07.318986 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c1206d6e-7a67-4804-80fd-37784b815d4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:07.320857 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1019/2029] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:07 2022] DELETE /identity/v3/projects/d29ff77d79f64e7cb98b5ef3ee37db6c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:46:07.326718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f150878-7d7c-4b9d-898f-c68d320babda None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:07.327765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f150878-7d7c-4b9d-898f-c68d320babda None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:07.328233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f150878-7d7c-4b9d-898f-c68d320babda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:07.360656 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-3f150878-7d7c-4b9d-898f-c68d320babda None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:07.363607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-3f150878-7d7c-4b9d-898f-c68d320babda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:07.402791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b52ba-1470-4e20-a359-dac233e42c8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:07.402791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b52ba-1470-4e20-a359-dac233e42c8d service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:07.402791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b52ba-1470-4e20-a359-dac233e42c8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:07.402791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b52ba-1470-4e20-a359-dac233e42c8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:07.403270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4e9b52ba-1470-4e20-a359-dac233e42c8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:07.460787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1020/2030] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:07.471792 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbee36f-3069-45ed-a5a3-b2a0731f33fb None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:07.472460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbee36f-3069-45ed-a5a3-b2a0731f33fb None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:07.472932 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbee36f-3069-45ed-a5a3-b2a0731f33fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:07.514647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-3dbee36f-3069-45ed-a5a3-b2a0731f33fb None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:07.516749 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-3dbee36f-3069-45ed-a5a3-b2a0731f33fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:07.520055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:07.531754 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1021/2031] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:07.538090 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbee36f-3069-45ed-a5a3-b2a0731f33fb None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:07.553256 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b52ba-1470-4e20-a359-dac233e42c8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:07.554940 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b52ba-1470-4e20-a359-dac233e42c8d service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:07.555856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbee36f-3069-45ed-a5a3-b2a0731f33fb None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:07.562379 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c36aa8-2c7f-4a4e-ba71-2bd54cf8f4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:07.563345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c36aa8-2c7f-4a4e-ba71-2bd54cf8f4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:07.563752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c36aa8-2c7f-4a4e-ba71-2bd54cf8f4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:07.564213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c36aa8-2c7f-4a4e-ba71-2bd54cf8f4f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:07.565024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c36aa8-2c7f-4a4e-ba71-2bd54cf8f4f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:46:07.565635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c36aa8-2c7f-4a4e-ba71-2bd54cf8f4f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:07.567407 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1011/2032] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:07.568475 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c36aa8-2c7f-4a4e-ba71-2bd54cf8f4f4 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:07.572445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:07.606952 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b52ba-1470-4e20-a359-dac233e42c8d service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:07.607723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1022/2033] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:46:07.622674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b52ba-1470-4e20-a359-dac233e42c8d service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:07.646622 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afd549-5650-43e9-bbd3-4f2ff383e29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:07.647304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afd549-5650-43e9-bbd3-4f2ff383e29e admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:07.647588 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afd549-5650-43e9-bbd3-4f2ff383e29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:07.647861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afd549-5650-43e9-bbd3-4f2ff383e29e admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:07.648530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afd549-5650-43e9-bbd3-4f2ff383e29e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:07.655350 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afd549-5650-43e9-bbd3-4f2ff383e29e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:07.727999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1012/2034] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:46:07 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:07.735239 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c36aa8-2c7f-4a4e-ba71-2bd54cf8f4f4 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:07.783217 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c36aa8-2c7f-4a4e-ba71-2bd54cf8f4f4 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:07.906286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefaa4a-9ed6-4a27-8838-ede111047244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:07.907262 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefaa4a-9ed6-4a27-8838-ede111047244 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:07.907764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefaa4a-9ed6-4a27-8838-ede111047244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:07.908046 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefaa4a-9ed6-4a27-8838-ede111047244 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:07.908736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefaa4a-9ed6-4a27-8838-ede111047244 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:07.924099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefaa4a-9ed6-4a27-8838-ede111047244 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:07.955222 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1023/2035] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:46:07 2022] POST /identity/v3/users => generated 287 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:07.959687 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afd549-5650-43e9-bbd3-4f2ff383e29e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:07.978318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afd549-5650-43e9-bbd3-4f2ff383e29e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:07.986897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f83f74-c2db-4568-ad75-d296351325e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:07.990332 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f83f74-c2db-4568-ad75-d296351325e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:07.990332 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f83f74-c2db-4568-ad75-d296351325e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:07.990332 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f83f74-c2db-4568-ad75-d296351325e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:07.997221 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f83f74-c2db-4568-ad75-d296351325e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:07.997221 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f83f74-c2db-4568-ad75-d296351325e7 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:08.015448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1013/2036] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:08.020249 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefaa4a-9ed6-4a27-8838-ede111047244 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:08.031590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefaa4a-9ed6-4a27-8838-ede111047244 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:08.036770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a43f34-fa7b-4290-b9a8-a39151048850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:08.037483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a43f34-fa7b-4290-b9a8-a39151048850 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:08.037755 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a43f34-fa7b-4290-b9a8-a39151048850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:08.038005 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a43f34-fa7b-4290-b9a8-a39151048850 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:08.038666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a43f34-fa7b-4290-b9a8-a39151048850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:08.040291 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a43f34-fa7b-4290-b9a8-a39151048850 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:08.077487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1024/2037] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:08.078835 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f83f74-c2db-4568-ad75-d296351325e7 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:08.090519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f83f74-c2db-4568-ad75-d296351325e7 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:08.096342 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ed74e-c1d8-48fa-adb0-ecfa9da5d4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:08.097052 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ed74e-c1d8-48fa-adb0-ecfa9da5d4f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:08.097464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ed74e-c1d8-48fa-adb0-ecfa9da5d4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:08.097777 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ed74e-c1d8-48fa-adb0-ecfa9da5d4f0 admin admin] PATH_INFO: `/v3/projects/658d000d0c994f9294ec1eb73f02aedf/users/e675cd97033f4f79bb657f9187ab3946/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:08.116906 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ed74e-c1d8-48fa-adb0-ecfa9da5d4f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=658d000d0c994f9294ec1eb73f02aedf, user_id=e675cd97033f4f79bb657f9187ab3946, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:08.119669 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ed74e-c1d8-48fa-adb0-ecfa9da5d4f0 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:08.139862 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1014/2038] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:08 2022] PUT /identity/v3/projects/658d000d0c994f9294ec1eb73f02aedf/users/e675cd97033f4f79bb657f9187ab3946/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:08.157773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fa00b-ea34-449a-94fa-a34980d4ed2d None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:08.157939 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fa00b-ea34-449a-94fa-a34980d4ed2d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:08.158283 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fa00b-ea34-449a-94fa-a34980d4ed2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:08.203110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-2f6fa00b-ea34-449a-94fa-a34980d4ed2d None None] MFA Rules not processed for user `e675cd97033f4f79bb657f9187ab3946`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:08.205097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2f6fa00b-ea34-449a-94fa-a34980d4ed2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:08.247194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ed74e-c1d8-48fa-adb0-ecfa9da5d4f0 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:08.249893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a23ed74e-c1d8-48fa-adb0-ecfa9da5d4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:08.292788 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1025/2039] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:08.305278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fa00b-ea34-449a-94fa-a34980d4ed2d None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:08.306988 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2f6fa00b-ea34-449a-94fa-a34980d4ed2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:08.338144 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ed74e-c1d8-48fa-adb0-ecfa9da5d4f0 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:08.342826 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeae47e-f6ec-496f-8de0-59765196f94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:08.343491 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeae47e-f6ec-496f-8de0-59765196f94d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:08.343769 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeae47e-f6ec-496f-8de0-59765196f94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:08.344163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeae47e-f6ec-496f-8de0-59765196f94d admin admin] PATH_INFO: `/v3/users/723f924b01a148efbeab1051015890a4` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:08.345499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeae47e-f6ec-496f-8de0-59765196f94d admin admin] RBAC: Authorizing `identity:delete_user(user_id=723f924b01a148efbeab1051015890a4)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:08.347552 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeae47e-f6ec-496f-8de0-59765196f94d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:08.379107 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fa00b-ea34-449a-94fa-a34980d4ed2d None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:08.384595 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e3f6b-aef3-4712-a075-695b7fffb6d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:08.384961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e3f6b-aef3-4712-a075-695b7fffb6d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:08.386021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e3f6b-aef3-4712-a075-695b7fffb6d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:08.386499 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e3f6b-aef3-4712-a075-695b7fffb6d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:08.401897 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e3f6b-aef3-4712-a075-695b7fffb6d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:08.404429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e3f6b-aef3-4712-a075-695b7fffb6d9 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:08.412234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-faeae47e-f6ec-496f-8de0-59765196f94d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '723f924b01a148efbeab1051015890a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:08.418310 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1026/2040] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:46:08.422814 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-faeae47e-f6ec-496f-8de0-59765196f94d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '723f924b01a148efbeab1051015890a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:08.424096 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-faeae47e-f6ec-496f-8de0-59765196f94d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '723f924b01a148efbeab1051015890a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:08.425123 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-faeae47e-f6ec-496f-8de0-59765196f94d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '723f924b01a148efbeab1051015890a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:08.447911 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e3f6b-aef3-4712-a075-695b7fffb6d9 service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:08.448980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-faeae47e-f6ec-496f-8de0-59765196f94d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '723f924b01a148efbeab1051015890a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:08.459072 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e3f6b-aef3-4712-a075-695b7fffb6d9 service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:08.464373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90debeeb-d984-4c02-85b8-d77241f077bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:08.465024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90debeeb-d984-4c02-85b8-d77241f077bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:08.465343 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90debeeb-d984-4c02-85b8-d77241f077bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:08.465624 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90debeeb-d984-4c02-85b8-d77241f077bb admin admin] PATH_INFO: `/v3/users/e675cd97033f4f79bb657f9187ab3946` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:08.466883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90debeeb-d984-4c02-85b8-d77241f077bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e675cd97033f4f79bb657f9187ab3946)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:08.468850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90debeeb-d984-4c02-85b8-d77241f077bb admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:08.469407 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:08.470822 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:08.476146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1015/2041] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:08 2022] DELETE /identity/v3/users/723f924b01a148efbeab1051015890a4 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:08.485006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed1eae-1a81-4f1f-9ebc-f962132ba75a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:08.487299 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed1eae-1a81-4f1f-9ebc-f962132ba75a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:08.488031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed1eae-1a81-4f1f-9ebc-f962132ba75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:08.533374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d8ed1eae-1a81-4f1f-9ebc-f962132ba75a None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:08.536090 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d8ed1eae-1a81-4f1f-9ebc-f962132ba75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:08.544193 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-90debeeb-d984-4c02-85b8-d77241f077bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e675cd97033f4f79bb657f9187ab3946'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:08.554516 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-90debeeb-d984-4c02-85b8-d77241f077bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e675cd97033f4f79bb657f9187ab3946'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:08.555153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-90debeeb-d984-4c02-85b8-d77241f077bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e675cd97033f4f79bb657f9187ab3946'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:08.555819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-90debeeb-d984-4c02-85b8-d77241f077bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e675cd97033f4f79bb657f9187ab3946'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:08.564493 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-90debeeb-d984-4c02-85b8-d77241f077bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e675cd97033f4f79bb657f9187ab3946'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:08.587668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1027/2042] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:08 2022] DELETE /identity/v3/users/e675cd97033f4f79bb657f9187ab3946 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:46:08.592914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b8771-07ae-4170-a37a-a37c22ddded9 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:08.593239 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b8771-07ae-4170-a37a-a37c22ddded9 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:08.593657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b8771-07ae-4170-a37a-a37c22ddded9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:08.614884 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-6f1b8771-07ae-4170-a37a-a37c22ddded9 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:08.616210 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6f1b8771-07ae-4170-a37a-a37c22ddded9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:08.621359 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1016/2043] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:08.632329 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed1eae-1a81-4f1f-9ebc-f962132ba75a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:08.634609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d8ed1eae-1a81-4f1f-9ebc-f962132ba75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:08.684182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1028/2044] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:08.695390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b8771-07ae-4170-a37a-a37c22ddded9 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:08.697315 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6f1b8771-07ae-4170-a37a-a37c22ddded9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:08.703786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed1eae-1a81-4f1f-9ebc-f962132ba75a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:08.761428 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd775aa-beec-4015-a925-72177a83e87d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:08.762698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd775aa-beec-4015-a925-72177a83e87d service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:08.762978 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd775aa-beec-4015-a925-72177a83e87d service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:08.763247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd775aa-beec-4015-a925-72177a83e87d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:08.763780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b8771-07ae-4170-a37a-a37c22ddded9 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:08.768171 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec26d66-fde6-4a3d-8355-3ab375a48e3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:08.768916 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec26d66-fde6-4a3d-8355-3ab375a48e3e service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:08.769482 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec26d66-fde6-4a3d-8355-3ab375a48e3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:08.769833 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec26d66-fde6-4a3d-8355-3ab375a48e3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:08.773646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd775aa-beec-4015-a925-72177a83e87d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:08.775097 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd775aa-beec-4015-a925-72177a83e87d service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:08.780950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec26d66-fde6-4a3d-8355-3ab375a48e3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:08.782489 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec26d66-fde6-4a3d-8355-3ab375a48e3e service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:08.785840 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1017/2045] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:46:08.798538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1029/2046] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:46:09.390346 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd775aa-beec-4015-a925-72177a83e87d service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:09.393487 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4dd775aa-beec-4015-a925-72177a83e87d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:09.415885 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd775aa-beec-4015-a925-72177a83e87d service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:09.422273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c47d5-c34f-4760-8cf5-d60e56b589d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:09.422562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c47d5-c34f-4760-8cf5-d60e56b589d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:09.422647 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c47d5-c34f-4760-8cf5-d60e56b589d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:09.422899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c47d5-c34f-4760-8cf5-d60e56b589d1 admin admin] PATH_INFO: `/v3/projects/658d000d0c994f9294ec1eb73f02aedf` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:09.425549 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c47d5-c34f-4760-8cf5-d60e56b589d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=658d000d0c994f9294ec1eb73f02aedf)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:09.427575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c47d5-c34f-4760-8cf5-d60e56b589d1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:09.514498 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec26d66-fde6-4a3d-8355-3ab375a48e3e service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:09.517541 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-cec26d66-fde6-4a3d-8355-3ab375a48e3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:09.524500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9d5c47d5-c34f-4760-8cf5-d60e56b589d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658d000d0c994f9294ec1eb73f02aedf', 'request_id': 'req-9d5c47d5-c34f-4760-8cf5-d60e56b589d1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:09.529542 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:09.538748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9d5c47d5-c34f-4760-8cf5-d60e56b589d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658d000d0c994f9294ec1eb73f02aedf', 'request_id': 'req-9d5c47d5-c34f-4760-8cf5-d60e56b589d1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:09.566155 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9d5c47d5-c34f-4760-8cf5-d60e56b589d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658d000d0c994f9294ec1eb73f02aedf', 'request_id': 'req-9d5c47d5-c34f-4760-8cf5-d60e56b589d1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:09.579779 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec26d66-fde6-4a3d-8355-3ab375a48e3e service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:09.592469 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9d5c47d5-c34f-4760-8cf5-d60e56b589d1 admin admin] The token cache is being invalidate because project 658d000d0c994f9294ec1eb73f02aedf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:09.592982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9d5c47d5-c34f-4760-8cf5-d60e56b589d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:09.597206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9d5c47d5-c34f-4760-8cf5-d60e56b589d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:09.606121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1018/2047] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:09 2022] DELETE /identity/v3/projects/658d000d0c994f9294ec1eb73f02aedf => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:09.612130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693b123-530f-429b-ba70-a3fea5e1ef87 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:09.612593 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693b123-530f-429b-ba70-a3fea5e1ef87 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:09.613105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693b123-530f-429b-ba70-a3fea5e1ef87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:09.644711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:09.656136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-2693b123-530f-429b-ba70-a3fea5e1ef87 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:09.658336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2693b123-530f-429b-ba70-a3fea5e1ef87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:09.736304 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a64b50-3ff0-447e-8b3a-b877c628ccd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:09.736962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a64b50-3ff0-447e-8b3a-b877c628ccd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:09.737278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a64b50-3ff0-447e-8b3a-b877c628ccd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:09.737566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a64b50-3ff0-447e-8b3a-b877c628ccd6 admin admin] PATH_INFO: `/v3/projects/fc2db39918144652925651a08d3bb2cf` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:09.738916 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a64b50-3ff0-447e-8b3a-b877c628ccd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc2db39918144652925651a08d3bb2cf)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:09.740889 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a64b50-3ff0-447e-8b3a-b877c628ccd6 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:09.757334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1019/2048] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:09.768377 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9236d-80fb-4679-ae3f-90fce0f22690 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:09.769007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9236d-80fb-4679-ae3f-90fce0f22690 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:09.769782 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9236d-80fb-4679-ae3f-90fce0f22690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:09.797763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f3d9236d-80fb-4679-ae3f-90fce0f22690 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:09.799053 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f3d9236d-80fb-4679-ae3f-90fce0f22690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:09.809335 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1020/2049] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:09.814844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9236d-80fb-4679-ae3f-90fce0f22690 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:09.826772 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9236d-80fb-4679-ae3f-90fce0f22690 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:09.843317 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-14a64b50-3ff0-447e-8b3a-b877c628ccd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc2db39918144652925651a08d3bb2cf', 'request_id': 'req-14a64b50-3ff0-447e-8b3a-b877c628ccd6'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:09.853843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-14a64b50-3ff0-447e-8b3a-b877c628ccd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc2db39918144652925651a08d3bb2cf', 'request_id': 'req-14a64b50-3ff0-447e-8b3a-b877c628ccd6'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:09.864265 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-14a64b50-3ff0-447e-8b3a-b877c628ccd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc2db39918144652925651a08d3bb2cf', 'request_id': 'req-14a64b50-3ff0-447e-8b3a-b877c628ccd6'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:09.883083 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-14a64b50-3ff0-447e-8b3a-b877c628ccd6 admin admin] The token cache is being invalidate because project fc2db39918144652925651a08d3bb2cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:09.883915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-14a64b50-3ff0-447e-8b3a-b877c628ccd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:09.885194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-14a64b50-3ff0-447e-8b3a-b877c628ccd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:09.890700 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1030/2050] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:09 2022] DELETE /identity/v3/projects/fc2db39918144652925651a08d3bb2cf => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:46:09.892619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a64b50-3ff0-447e-8b3a-b877c628ccd6 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:09.895217 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-14a64b50-3ff0-447e-8b3a-b877c628ccd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:09.922176 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4203b-1a59-49a9-93d2-1a009cb291f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:09.922908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4203b-1a59-49a9-93d2-1a009cb291f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:09.923249 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4203b-1a59-49a9-93d2-1a009cb291f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:09.923582 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4203b-1a59-49a9-93d2-1a009cb291f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:09.924321 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b4203b-1a59-49a9-93d2-1a009cb291f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:46:09.924684 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b4203b-1a59-49a9-93d2-1a009cb291f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:09.926524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b4203b-1a59-49a9-93d2-1a009cb291f1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:09.945269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1021/2051] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:46:09.951999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4203b-1a59-49a9-93d2-1a009cb291f1 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:09.955868 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-69b4203b-1a59-49a9-93d2-1a009cb291f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:09.982186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4203b-1a59-49a9-93d2-1a009cb291f1 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:09.990741 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3592f55-0709-45b3-9e43-61b7ea7145ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:09.991819 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3592f55-0709-45b3-9e43-61b7ea7145ec admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:09.992272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3592f55-0709-45b3-9e43-61b7ea7145ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:09.992729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3592f55-0709-45b3-9e43-61b7ea7145ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:09.994080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3592f55-0709-45b3-9e43-61b7ea7145ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:09.999034 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a64b50-3ff0-447e-8b3a-b877c628ccd6 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:09.999284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3592f55-0709-45b3-9e43-61b7ea7145ec admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:10.004995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec27ff1-1aeb-47d4-9741-bab0e5d4b2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:10.005869 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec27ff1-1aeb-47d4-9741-bab0e5d4b2fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:10.006356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec27ff1-1aeb-47d4-9741-bab0e5d4b2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:10.006724 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec27ff1-1aeb-47d4-9741-bab0e5d4b2fe admin admin] PATH_INFO: `/v3/users/1a7c89730c034d3eb58a7cf6cac2bd56` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:10.009027 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec27ff1-1aeb-47d4-9741-bab0e5d4b2fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a7c89730c034d3eb58a7cf6cac2bd56)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:10.011363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec27ff1-1aeb-47d4-9741-bab0e5d4b2fe admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:10.076202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1022/2052] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:46:09 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:10.078294 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3592f55-0709-45b3-9e43-61b7ea7145ec admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:10.100607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-2ec27ff1-1aeb-47d4-9741-bab0e5d4b2fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a7c89730c034d3eb58a7cf6cac2bd56'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:10.120608 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-2ec27ff1-1aeb-47d4-9741-bab0e5d4b2fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a7c89730c034d3eb58a7cf6cac2bd56'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:10.122907 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-2ec27ff1-1aeb-47d4-9741-bab0e5d4b2fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a7c89730c034d3eb58a7cf6cac2bd56'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:10.125735 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-2ec27ff1-1aeb-47d4-9741-bab0e5d4b2fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a7c89730c034d3eb58a7cf6cac2bd56'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:10.129843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:10.140688 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-2ec27ff1-1aeb-47d4-9741-bab0e5d4b2fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a7c89730c034d3eb58a7cf6cac2bd56'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:10.170841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:10.193569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1031/2053] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:09 2022] DELETE /identity/v3/users/1a7c89730c034d3eb58a7cf6cac2bd56 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:10.198871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3592f55-0709-45b3-9e43-61b7ea7145ec admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:10.339466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740fea38-2213-4e18-85c5-00c20b166717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:10.339944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fea38-2213-4e18-85c5-00c20b166717 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:10.339944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fea38-2213-4e18-85c5-00c20b166717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:10.340278 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fea38-2213-4e18-85c5-00c20b166717 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:10.341348 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740fea38-2213-4e18-85c5-00c20b166717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:10.348677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740fea38-2213-4e18-85c5-00c20b166717 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:10.436261 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1023/2054] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:46:10 2022] POST /identity/v3/users => generated 287 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:10.437456 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec27ff1-1aeb-47d4-9741-bab0e5d4b2fe admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:10.440380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2ec27ff1-1aeb-47d4-9741-bab0e5d4b2fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:10.480646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec27ff1-1aeb-47d4-9741-bab0e5d4b2fe admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:10.487124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacab6e-cb9c-44d6-a938-bc902833f137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:10.488094 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacab6e-cb9c-44d6-a938-bc902833f137 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:10.488675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacab6e-cb9c-44d6-a938-bc902833f137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:10.489119 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacab6e-cb9c-44d6-a938-bc902833f137 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:10.490055 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eacab6e-cb9c-44d6-a938-bc902833f137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:10.492017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eacab6e-cb9c-44d6-a938-bc902833f137 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:10.508399 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1032/2055] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:10.512513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740fea38-2213-4e18-85c5-00c20b166717 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:10.532158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740fea38-2213-4e18-85c5-00c20b166717 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:10.539063 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886face4-ec7e-4837-9f25-852d478d5ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:10.540834 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886face4-ec7e-4837-9f25-852d478d5ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:10.541304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886face4-ec7e-4837-9f25-852d478d5ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:10.541731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886face4-ec7e-4837-9f25-852d478d5ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:10.542572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886face4-ec7e-4837-9f25-852d478d5ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:10.544874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886face4-ec7e-4837-9f25-852d478d5ea9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:10.564512 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1024/2056] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:46:10.565972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacab6e-cb9c-44d6-a938-bc902833f137 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:10.591554 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacab6e-cb9c-44d6-a938-bc902833f137 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:10.608141 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54850cb6-cb29-4318-8d36-13618257129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:10.609759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54850cb6-cb29-4318-8d36-13618257129f admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:10.610285 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54850cb6-cb29-4318-8d36-13618257129f admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:10.610738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54850cb6-cb29-4318-8d36-13618257129f admin admin] PATH_INFO: `/v3/projects/9c05c9127854481aa2ff754f336e7895/users/97028c1b3fe141dea0fa530a4b8aff30/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:10.676712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54850cb6-cb29-4318-8d36-13618257129f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c05c9127854481aa2ff754f336e7895, user_id=97028c1b3fe141dea0fa530a4b8aff30, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:10.686111 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54850cb6-cb29-4318-8d36-13618257129f admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:10.732826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1033/2057] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:10 2022] PUT /identity/v3/projects/9c05c9127854481aa2ff754f336e7895/users/97028c1b3fe141dea0fa530a4b8aff30/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:46:10.745103 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ab9d8-c355-411b-9efe-231030e659dd None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:10.745681 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ab9d8-c355-411b-9efe-231030e659dd None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:10.746269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ab9d8-c355-411b-9efe-231030e659dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:10.796276 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-021ab9d8-c355-411b-9efe-231030e659dd None None] MFA Rules not processed for user `97028c1b3fe141dea0fa530a4b8aff30`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:10.797619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-021ab9d8-c355-411b-9efe-231030e659dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:10.908472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1025/2058] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:10.948412 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54850cb6-cb29-4318-8d36-13618257129f admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:10.956648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-54850cb6-cb29-4318-8d36-13618257129f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:10.966214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ab9d8-c355-411b-9efe-231030e659dd None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:10.967842 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-021ab9d8-c355-411b-9efe-231030e659dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:11.104664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54850cb6-cb29-4318-8d36-13618257129f admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:11.105688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ab9d8-c355-411b-9efe-231030e659dd None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:11.110753 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317b399-e80a-4139-b6ec-a654fe079c27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:11.112049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317b399-e80a-4139-b6ec-a654fe079c27 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:11.112770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317b399-e80a-4139-b6ec-a654fe079c27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:11.113951 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317b399-e80a-4139-b6ec-a654fe079c27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:11.115054 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9540b-ad6c-490f-ad53-1083e1c6865a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:11.116713 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9540b-ad6c-490f-ad53-1083e1c6865a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:11.117093 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9540b-ad6c-490f-ad53-1083e1c6865a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:11.117896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9540b-ad6c-490f-ad53-1083e1c6865a admin admin] PATH_INFO: `/v3/projects/2ea435e53389452bb323644295c1b296` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:11.120027 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9540b-ad6c-490f-ad53-1083e1c6865a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ea435e53389452bb323644295c1b296)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:11.122193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9540b-ad6c-490f-ad53-1083e1c6865a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:11.125686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:11.130841 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317b399-e80a-4139-b6ec-a654fe079c27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:11.133859 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317b399-e80a-4139-b6ec-a654fe079c27 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:11.167313 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1034/2059] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:11.239270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-76e9540b-ad6c-490f-ad53-1083e1c6865a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ea435e53389452bb323644295c1b296', 'request_id': 'req-76e9540b-ad6c-490f-ad53-1083e1c6865a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:11.251430 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-76e9540b-ad6c-490f-ad53-1083e1c6865a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ea435e53389452bb323644295c1b296', 'request_id': 'req-76e9540b-ad6c-490f-ad53-1083e1c6865a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:11.273522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-76e9540b-ad6c-490f-ad53-1083e1c6865a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ea435e53389452bb323644295c1b296', 'request_id': 'req-76e9540b-ad6c-490f-ad53-1083e1c6865a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:11.306488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-76e9540b-ad6c-490f-ad53-1083e1c6865a admin admin] The token cache is being invalidate because project 2ea435e53389452bb323644295c1b296 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:11.306969 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-76e9540b-ad6c-490f-ad53-1083e1c6865a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:11.312201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-76e9540b-ad6c-490f-ad53-1083e1c6865a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:11.315236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1026/2060] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:10 2022] DELETE /identity/v3/projects/2ea435e53389452bb323644295c1b296 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:11.322357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd1726-a504-4123-b604-347390d5fca4 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:11.322716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd1726-a504-4123-b604-347390d5fca4 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:11.323024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd1726-a504-4123-b604-347390d5fca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:11.362636 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-b2dd1726-a504-4123-b604-347390d5fca4 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:11.365365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-b2dd1726-a504-4123-b604-347390d5fca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:11.489182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1035/2061] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:11.502110 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1207f52-dc23-4cd4-b6fb-9e99034cf713 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:11.502853 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1207f52-dc23-4cd4-b6fb-9e99034cf713 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:11.503017 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1207f52-dc23-4cd4-b6fb-9e99034cf713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:11.533438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b1207f52-dc23-4cd4-b6fb-9e99034cf713 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:11.534956 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b1207f52-dc23-4cd4-b6fb-9e99034cf713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:11.550185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1027/2062] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:11.552202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd1726-a504-4123-b604-347390d5fca4 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:11.569623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd1726-a504-4123-b604-347390d5fca4 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:11.579755 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2c22b-5f1d-46d9-befe-64e20d1d73d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:11.580637 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2c22b-5f1d-46d9-befe-64e20d1d73d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:11.580918 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2c22b-5f1d-46d9-befe-64e20d1d73d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:11.581258 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2c22b-5f1d-46d9-befe-64e20d1d73d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:11.582041 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2c22b-5f1d-46d9-befe-64e20d1d73d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:46:11.582381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2c22b-5f1d-46d9-befe-64e20d1d73d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:11.584184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2c22b-5f1d-46d9-befe-64e20d1d73d3 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:11.602737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1036/2063] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:46:11.604655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1207f52-dc23-4cd4-b6fb-9e99034cf713 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:11.639756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1207f52-dc23-4cd4-b6fb-9e99034cf713 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:11.660477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30a427-5555-4bbc-b0b2-f4e4f0de1bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:11.661666 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30a427-5555-4bbc-b0b2-f4e4f0de1bfb admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:11.662367 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30a427-5555-4bbc-b0b2-f4e4f0de1bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:11.663007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30a427-5555-4bbc-b0b2-f4e4f0de1bfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:11.664194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30a427-5555-4bbc-b0b2-f4e4f0de1bfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:11.682799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30a427-5555-4bbc-b0b2-f4e4f0de1bfb admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:11.785911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1028/2064] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:46:11 2022] POST /identity/v3/projects => generated 383 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:11.787895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2c22b-5f1d-46d9-befe-64e20d1d73d3 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:11.803172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2c22b-5f1d-46d9-befe-64e20d1d73d3 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:11.836845 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:11.878750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e852e2d-11a3-4b09-8577-375f92950be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:11.880064 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e852e2d-11a3-4b09-8577-375f92950be9 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:11.880064 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e852e2d-11a3-4b09-8577-375f92950be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:11.880064 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e852e2d-11a3-4b09-8577-375f92950be9 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:11.880663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e852e2d-11a3-4b09-8577-375f92950be9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:11.882693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e852e2d-11a3-4b09-8577-375f92950be9 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:11.910452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1037/2065] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:46:11 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:46:11.914643 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30a427-5555-4bbc-b0b2-f4e4f0de1bfb admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:11.926407 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30a427-5555-4bbc-b0b2-f4e4f0de1bfb admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:11.933478 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6e302-731d-4516-8d68-9de57ccf8da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:11.934086 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6e302-731d-4516-8d68-9de57ccf8da8 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:11.934242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6e302-731d-4516-8d68-9de57ccf8da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:11.934502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6e302-731d-4516-8d68-9de57ccf8da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:11.935273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6e302-731d-4516-8d68-9de57ccf8da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:11.936838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6e302-731d-4516-8d68-9de57ccf8da8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:11.948022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1029/2066] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:11.952221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e852e2d-11a3-4b09-8577-375f92950be9 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:11.964212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e852e2d-11a3-4b09-8577-375f92950be9 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:11.970524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbeace4-8908-47c9-a4bc-dbd72c22a7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:11.971230 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeace4-8908-47c9-a4bc-dbd72c22a7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:11.971611 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeace4-8908-47c9-a4bc-dbd72c22a7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:11.974785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeace4-8908-47c9-a4bc-dbd72c22a7e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:11.974785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbeace4-8908-47c9-a4bc-dbd72c22a7e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:11.974785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbeace4-8908-47c9-a4bc-dbd72c22a7e9 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:12.000648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1038/2067] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:12.006577 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6e302-731d-4516-8d68-9de57ccf8da8 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:12.031821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6e302-731d-4516-8d68-9de57ccf8da8 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:12.039174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d01e6-e48f-4b66-9776-ba2f5dddbc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:12.040341 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d01e6-e48f-4b66-9776-ba2f5dddbc6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:12.040938 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d01e6-e48f-4b66-9776-ba2f5dddbc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:12.041470 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d01e6-e48f-4b66-9776-ba2f5dddbc6d admin admin] PATH_INFO: `/v3/projects/bfd5117f6fec43799d8d814e0daafe99/users/676f94336de149c1abd0db896bafb62d/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:12.066621 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d01e6-e48f-4b66-9776-ba2f5dddbc6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd5117f6fec43799d8d814e0daafe99, user_id=676f94336de149c1abd0db896bafb62d, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:12.072223 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d01e6-e48f-4b66-9776-ba2f5dddbc6d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:12.096621 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1030/2068] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:12 2022] PUT /identity/v3/projects/bfd5117f6fec43799d8d814e0daafe99/users/676f94336de149c1abd0db896bafb62d/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:12.112770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beef5ce-340c-4592-a8d5-d22340b95cda None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:12.113410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beef5ce-340c-4592-a8d5-d22340b95cda None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:12.113696 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beef5ce-340c-4592-a8d5-d22340b95cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:12.157728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-7beef5ce-340c-4592-a8d5-d22340b95cda None None] MFA Rules not processed for user `676f94336de149c1abd0db896bafb62d`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:12.160086 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7beef5ce-340c-4592-a8d5-d22340b95cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:12.328741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1039/2069] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:12.335322 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d01e6-e48f-4b66-9776-ba2f5dddbc6d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:12.337548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-767d01e6-e48f-4b66-9776-ba2f5dddbc6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:12.427578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d01e6-e48f-4b66-9776-ba2f5dddbc6d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:12.433424 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c363a6f-81ea-41ce-9547-ef9476e97bce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:12.433875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c363a6f-81ea-41ce-9547-ef9476e97bce service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:12.434332 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c363a6f-81ea-41ce-9547-ef9476e97bce service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:12.434623 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c363a6f-81ea-41ce-9547-ef9476e97bce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:12.446297 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c363a6f-81ea-41ce-9547-ef9476e97bce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:12.447836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c363a6f-81ea-41ce-9547-ef9476e97bce service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:12.460441 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1031/2070] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:12.485311 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beef5ce-340c-4592-a8d5-d22340b95cda None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:12.623070 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beef5ce-340c-4592-a8d5-d22340b95cda None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:12.630377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d680bb-f7b4-43fb-8989-a0c622095e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:12.631051 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d680bb-f7b4-43fb-8989-a0c622095e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:12.631475 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d680bb-f7b4-43fb-8989-a0c622095e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:12.631627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d680bb-f7b4-43fb-8989-a0c622095e88 admin admin] PATH_INFO: `/v3/users/676f94336de149c1abd0db896bafb62d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:12.633534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d680bb-f7b4-43fb-8989-a0c622095e88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=676f94336de149c1abd0db896bafb62d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:12.638500 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d680bb-f7b4-43fb-8989-a0c622095e88 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:12.714462 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:12.731178 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f7d680bb-f7b4-43fb-8989-a0c622095e88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '676f94336de149c1abd0db896bafb62d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:12.746102 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f7d680bb-f7b4-43fb-8989-a0c622095e88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '676f94336de149c1abd0db896bafb62d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:12.747131 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f7d680bb-f7b4-43fb-8989-a0c622095e88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '676f94336de149c1abd0db896bafb62d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:12.748522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f7d680bb-f7b4-43fb-8989-a0c622095e88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '676f94336de149c1abd0db896bafb62d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:12.763154 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f7d680bb-f7b4-43fb-8989-a0c622095e88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '676f94336de149c1abd0db896bafb62d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:12.798476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1040/2071] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:12 2022] DELETE /identity/v3/users/676f94336de149c1abd0db896bafb62d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:12.807739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b39342-4404-4341-88ae-e3e75a30da80 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:12.809328 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b39342-4404-4341-88ae-e3e75a30da80 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:12.810053 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b39342-4404-4341-88ae-e3e75a30da80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:12.852777 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d2b39342-4404-4341-88ae-e3e75a30da80 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:12.854512 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d2b39342-4404-4341-88ae-e3e75a30da80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:12.919762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:13.008889 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1032/2072] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:13.023321 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d680bb-f7b4-43fb-8989-a0c622095e88 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:13.024850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f7d680bb-f7b4-43fb-8989-a0c622095e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:13.126209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d680bb-f7b4-43fb-8989-a0c622095e88 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:13.130786 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd7bcb-32c5-4514-8205-94f985fc5069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:13.132228 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd7bcb-32c5-4514-8205-94f985fc5069 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:13.132228 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd7bcb-32c5-4514-8205-94f985fc5069 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:13.132228 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd7bcb-32c5-4514-8205-94f985fc5069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:13.145398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd7bcb-32c5-4514-8205-94f985fc5069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:13.146949 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd7bcb-32c5-4514-8205-94f985fc5069 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:13.158375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1041/2073] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:13.594526 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b39342-4404-4341-88ae-e3e75a30da80 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:13.596256 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d2b39342-4404-4341-88ae-e3e75a30da80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:13.612580 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b39342-4404-4341-88ae-e3e75a30da80 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:13.625793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522b519-0b95-4821-9aa6-98ee0d7fd138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:13.626286 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522b519-0b95-4821-9aa6-98ee0d7fd138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:13.626606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522b519-0b95-4821-9aa6-98ee0d7fd138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:13.626954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522b519-0b95-4821-9aa6-98ee0d7fd138 admin admin] PATH_INFO: `/v3/projects/bfd5117f6fec43799d8d814e0daafe99` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:13.628065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522b519-0b95-4821-9aa6-98ee0d7fd138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfd5117f6fec43799d8d814e0daafe99)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:13.630241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522b519-0b95-4821-9aa6-98ee0d7fd138 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:13.695687 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c522b519-0b95-4821-9aa6-98ee0d7fd138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfd5117f6fec43799d8d814e0daafe99', 'request_id': 'req-c522b519-0b95-4821-9aa6-98ee0d7fd138'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:13.705821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c522b519-0b95-4821-9aa6-98ee0d7fd138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfd5117f6fec43799d8d814e0daafe99', 'request_id': 'req-c522b519-0b95-4821-9aa6-98ee0d7fd138'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:13.718234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c522b519-0b95-4821-9aa6-98ee0d7fd138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfd5117f6fec43799d8d814e0daafe99', 'request_id': 'req-c522b519-0b95-4821-9aa6-98ee0d7fd138'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:13.742731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c522b519-0b95-4821-9aa6-98ee0d7fd138 admin admin] The token cache is being invalidate because project bfd5117f6fec43799d8d814e0daafe99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:13.743043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c522b519-0b95-4821-9aa6-98ee0d7fd138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:13.744138 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c522b519-0b95-4821-9aa6-98ee0d7fd138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:13.747037 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1033/2074] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:13 2022] DELETE /identity/v3/projects/bfd5117f6fec43799d8d814e0daafe99 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:13.754958 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767963f8-6a75-4ecf-8267-353e66043463 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:13.755695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767963f8-6a75-4ecf-8267-353e66043463 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:13.755834 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767963f8-6a75-4ecf-8267-353e66043463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:13.796786 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-767963f8-6a75-4ecf-8267-353e66043463 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:13.799110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-767963f8-6a75-4ecf-8267-353e66043463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:13.888521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1042/2075] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:13.899356 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781260a-4aec-42d0-8308-4f8b01ea5e30 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:13.899876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781260a-4aec-42d0-8308-4f8b01ea5e30 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:13.900354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781260a-4aec-42d0-8308-4f8b01ea5e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:13.928932 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-0781260a-4aec-42d0-8308-4f8b01ea5e30 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:13.930245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0781260a-4aec-42d0-8308-4f8b01ea5e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:13.939330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1034/2076] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:13.946909 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767963f8-6a75-4ecf-8267-353e66043463 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:13.960914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767963f8-6a75-4ecf-8267-353e66043463 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:13.967256 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94faae2-a99e-4b17-a16c-e0cda2396323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:13.968263 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94faae2-a99e-4b17-a16c-e0cda2396323 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:13.968896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94faae2-a99e-4b17-a16c-e0cda2396323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:13.969008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94faae2-a99e-4b17-a16c-e0cda2396323 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:13.969722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94faae2-a99e-4b17-a16c-e0cda2396323 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:46:13.970048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94faae2-a99e-4b17-a16c-e0cda2396323 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:13.972912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94faae2-a99e-4b17-a16c-e0cda2396323 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:13.996429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1043/2077] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:46:14.001818 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781260a-4aec-42d0-8308-4f8b01ea5e30 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:14.013631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781260a-4aec-42d0-8308-4f8b01ea5e30 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:14.020854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e485c16-96c1-4cef-822d-2f611b5d6574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:14.021863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e485c16-96c1-4cef-822d-2f611b5d6574 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:14.022325 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e485c16-96c1-4cef-822d-2f611b5d6574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:14.022754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e485c16-96c1-4cef-822d-2f611b5d6574 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:14.023886 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e485c16-96c1-4cef-822d-2f611b5d6574 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:14.026642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e485c16-96c1-4cef-822d-2f611b5d6574 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:14.049075 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6fa34-2f13-4f13-86bf-bc4d83ae420a None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:14.049584 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6fa34-2f13-4f13-86bf-bc4d83ae420a None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:14.050057 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6fa34-2f13-4f13-86bf-bc4d83ae420a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:14.078981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-ddd6fa34-2f13-4f13-86bf-bc4d83ae420a None None] MFA Rules not processed for user `97028c1b3fe141dea0fa530a4b8aff30`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:14.080415 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ddd6fa34-2f13-4f13-86bf-bc4d83ae420a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:14.086507 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1035/2078] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:46:14 2022] POST /identity/v3/projects => generated 359 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:14.091408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e485c16-96c1-4cef-822d-2f611b5d6574 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:14.136784 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:14.156241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1044/2079] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:14.190560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e485c16-96c1-4cef-822d-2f611b5d6574 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:14.200839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633e66b-8844-4b47-9bed-3a9df4a4299b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:14.201684 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633e66b-8844-4b47-9bed-3a9df4a4299b admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:14.202082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633e66b-8844-4b47-9bed-3a9df4a4299b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:14.202610 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633e66b-8844-4b47-9bed-3a9df4a4299b admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:14.203517 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633e66b-8844-4b47-9bed-3a9df4a4299b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:14.206010 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633e66b-8844-4b47-9bed-3a9df4a4299b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:14.246611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1036/2080] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:46:14 2022] POST /identity/v3/users => generated 280 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:14.246842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6fa34-2f13-4f13-86bf-bc4d83ae420a None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:14.258674 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6fa34-2f13-4f13-86bf-bc4d83ae420a None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:14.264409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14ee7f-3a1e-401b-9217-d6a817b829e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:14.264955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14ee7f-3a1e-401b-9217-d6a817b829e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:14.265420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14ee7f-3a1e-401b-9217-d6a817b829e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:14.265695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14ee7f-3a1e-401b-9217-d6a817b829e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:14.266364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14ee7f-3a1e-401b-9217-d6a817b829e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:14.267999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14ee7f-3a1e-401b-9217-d6a817b829e7 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:14.280640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1045/2081] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:46:14.284677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633e66b-8844-4b47-9bed-3a9df4a4299b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:14.296908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633e66b-8844-4b47-9bed-3a9df4a4299b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:14.303536 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d52c42-ae86-4c2a-babe-1ce455ba257c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:14.304597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d52c42-ae86-4c2a-babe-1ce455ba257c admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:14.305095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d52c42-ae86-4c2a-babe-1ce455ba257c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:14.305659 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d52c42-ae86-4c2a-babe-1ce455ba257c admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:14.306708 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d52c42-ae86-4c2a-babe-1ce455ba257c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:14.309467 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d52c42-ae86-4c2a-babe-1ce455ba257c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:14.329983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1037/2082] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:14.336132 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14ee7f-3a1e-401b-9217-d6a817b829e7 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:14.356155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14ee7f-3a1e-401b-9217-d6a817b829e7 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:14.364617 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8511e47-fe59-4d0f-8c1d-f32c6491ad60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:14.365748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8511e47-fe59-4d0f-8c1d-f32c6491ad60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:14.366216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8511e47-fe59-4d0f-8c1d-f32c6491ad60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:14.366699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8511e47-fe59-4d0f-8c1d-f32c6491ad60 admin admin] PATH_INFO: `/v3/projects/0c67efde8f1b4bec9b69f2db261646cd/users/91917c32a6dc4807b79678ba0de973a7/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:14.397390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8511e47-fe59-4d0f-8c1d-f32c6491ad60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c67efde8f1b4bec9b69f2db261646cd, user_id=91917c32a6dc4807b79678ba0de973a7, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:14.404185 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8511e47-fe59-4d0f-8c1d-f32c6491ad60 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:14.437277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1046/2083] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:14 2022] PUT /identity/v3/projects/0c67efde8f1b4bec9b69f2db261646cd/users/91917c32a6dc4807b79678ba0de973a7/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:14.451371 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed49d9-c728-4612-b72e-544d211e2dfb None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:14.451791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed49d9-c728-4612-b72e-544d211e2dfb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:14.452275 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed49d9-c728-4612-b72e-544d211e2dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:14.466951 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:14.505706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-6aed49d9-c728-4612-b72e-544d211e2dfb None None] MFA Rules not processed for user `91917c32a6dc4807b79678ba0de973a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:14.507981 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6aed49d9-c728-4612-b72e-544d211e2dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:14.610796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1038/2084] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:14 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:14.687269 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8511e47-fe59-4d0f-8c1d-f32c6491ad60 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:14.690388 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-e8511e47-fe59-4d0f-8c1d-f32c6491ad60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:14.835932 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8511e47-fe59-4d0f-8c1d-f32c6491ad60 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:14.845262 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4052b6-efce-4877-bc12-b646a4e7dca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:14.846245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4052b6-efce-4877-bc12-b646a4e7dca2 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:14.846741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4052b6-efce-4877-bc12-b646a4e7dca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:14.847208 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4052b6-efce-4877-bc12-b646a4e7dca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:14.858461 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4052b6-efce-4877-bc12-b646a4e7dca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:14.861298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4052b6-efce-4877-bc12-b646a4e7dca2 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:14.877227 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1047/2085] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:14 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:14.892112 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed49d9-c728-4612-b72e-544d211e2dfb None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:14.972380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed49d9-c728-4612-b72e-544d211e2dfb None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:14.977105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09b338-487c-495b-9bd5-b20a8c547030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:14.977907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09b338-487c-495b-9bd5-b20a8c547030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:14.978861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09b338-487c-495b-9bd5-b20a8c547030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:14.978861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09b338-487c-495b-9bd5-b20a8c547030 admin admin] PATH_INFO: `/v3/users/91917c32a6dc4807b79678ba0de973a7` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:14.979817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09b338-487c-495b-9bd5-b20a8c547030 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91917c32a6dc4807b79678ba0de973a7)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:14.981908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09b338-487c-495b-9bd5-b20a8c547030 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:15.057285 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-1b09b338-487c-495b-9bd5-b20a8c547030 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91917c32a6dc4807b79678ba0de973a7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:15.071171 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-1b09b338-487c-495b-9bd5-b20a8c547030 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91917c32a6dc4807b79678ba0de973a7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:15.072090 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-1b09b338-487c-495b-9bd5-b20a8c547030 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91917c32a6dc4807b79678ba0de973a7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:15.073347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-1b09b338-487c-495b-9bd5-b20a8c547030 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91917c32a6dc4807b79678ba0de973a7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:15.098364 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-1b09b338-487c-495b-9bd5-b20a8c547030 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91917c32a6dc4807b79678ba0de973a7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:15.120372 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1039/2086] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:14 2022] DELETE /identity/v3/users/91917c32a6dc4807b79678ba0de973a7 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:15.123398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153ae3e-1e17-43be-922c-2bc063e6aeb3 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:15.123956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153ae3e-1e17-43be-922c-2bc063e6aeb3 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:15.125295 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153ae3e-1e17-43be-922c-2bc063e6aeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:15.150821 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-6153ae3e-1e17-43be-922c-2bc063e6aeb3 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:15.152326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6153ae3e-1e17-43be-922c-2bc063e6aeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:15.244062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1048/2087] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:15.256516 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09b338-487c-495b-9bd5-b20a8c547030 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:15.260612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1b09b338-487c-495b-9bd5-b20a8c547030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:15.288108 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:15.390170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09b338-487c-495b-9bd5-b20a8c547030 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:15.398037 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3d1e2-2237-4e6e-b86b-261221581be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:15.399055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3d1e2-2237-4e6e-b86b-261221581be7 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:15.399449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3d1e2-2237-4e6e-b86b-261221581be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:15.399736 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3d1e2-2237-4e6e-b86b-261221581be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:15.417541 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3d1e2-2237-4e6e-b86b-261221581be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:15.420411 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3d1e2-2237-4e6e-b86b-261221581be7 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:15.438322 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1040/2088] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:15.909672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153ae3e-1e17-43be-922c-2bc063e6aeb3 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:15.911820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6153ae3e-1e17-43be-922c-2bc063e6aeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:15.951452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153ae3e-1e17-43be-922c-2bc063e6aeb3 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:15.993358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a357f9-0de3-4589-b488-eff0d036cb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:15.994019 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a357f9-0de3-4589-b488-eff0d036cb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:15.994303 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a357f9-0de3-4589-b488-eff0d036cb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:15.994599 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a357f9-0de3-4589-b488-eff0d036cb58 admin admin] PATH_INFO: `/v3/projects/0c67efde8f1b4bec9b69f2db261646cd` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:15.996320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a357f9-0de3-4589-b488-eff0d036cb58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c67efde8f1b4bec9b69f2db261646cd)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:15.999448 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a357f9-0de3-4589-b488-eff0d036cb58 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:16.089945 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f5a357f9-0de3-4589-b488-eff0d036cb58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c67efde8f1b4bec9b69f2db261646cd', 'request_id': 'req-f5a357f9-0de3-4589-b488-eff0d036cb58'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:16.094280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:16.103035 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f5a357f9-0de3-4589-b488-eff0d036cb58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c67efde8f1b4bec9b69f2db261646cd', 'request_id': 'req-f5a357f9-0de3-4589-b488-eff0d036cb58'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:16.120750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f5a357f9-0de3-4589-b488-eff0d036cb58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c67efde8f1b4bec9b69f2db261646cd', 'request_id': 'req-f5a357f9-0de3-4589-b488-eff0d036cb58'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:16.168780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f5a357f9-0de3-4589-b488-eff0d036cb58 admin admin] The token cache is being invalidate because project 0c67efde8f1b4bec9b69f2db261646cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:16.171988 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f5a357f9-0de3-4589-b488-eff0d036cb58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:16.173482 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f5a357f9-0de3-4589-b488-eff0d036cb58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:16.178362 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1049/2089] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:15 2022] DELETE /identity/v3/projects/0c67efde8f1b4bec9b69f2db261646cd => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:16.183847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455c578-bb52-4ba1-b52d-08b6dd510e94 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:16.184999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455c578-bb52-4ba1-b52d-08b6dd510e94 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:16.185607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455c578-bb52-4ba1-b52d-08b6dd510e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:16.213903 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-4455c578-bb52-4ba1-b52d-08b6dd510e94 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:16.215566 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4455c578-bb52-4ba1-b52d-08b6dd510e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:16.309189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1041/2090] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:16.314414 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be5f6d-229f-4ef3-93bb-8c9abd4c6ca8 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:16.315027 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be5f6d-229f-4ef3-93bb-8c9abd4c6ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:16.315639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be5f6d-229f-4ef3-93bb-8c9abd4c6ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:16.355238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-67be5f6d-229f-4ef3-93bb-8c9abd4c6ca8 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:16.356565 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-67be5f6d-229f-4ef3-93bb-8c9abd4c6ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:16.377833 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1050/2091] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:16.383407 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455c578-bb52-4ba1-b52d-08b6dd510e94 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:16.395313 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455c578-bb52-4ba1-b52d-08b6dd510e94 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:16.401205 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd698cbb-35b0-4967-b905-08cf76d6df21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:16.401825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd698cbb-35b0-4967-b905-08cf76d6df21 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:16.402224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd698cbb-35b0-4967-b905-08cf76d6df21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:16.402511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd698cbb-35b0-4967-b905-08cf76d6df21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:16.403174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd698cbb-35b0-4967-b905-08cf76d6df21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:46:16.403507 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd698cbb-35b0-4967-b905-08cf76d6df21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:16.405519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd698cbb-35b0-4967-b905-08cf76d6df21 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:16.426408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1042/2092] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:46:16.433335 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be5f6d-229f-4ef3-93bb-8c9abd4c6ca8 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:16.455482 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be5f6d-229f-4ef3-93bb-8c9abd4c6ca8 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:16.466834 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f5138-68e3-4c9b-946d-f82fc7515716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:16.467942 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f5138-68e3-4c9b-946d-f82fc7515716 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:16.468415 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f5138-68e3-4c9b-946d-f82fc7515716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:16.468760 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f5138-68e3-4c9b-946d-f82fc7515716 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:16.469944 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f5138-68e3-4c9b-946d-f82fc7515716 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:16.479488 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f5138-68e3-4c9b-946d-f82fc7515716 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:16.557198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1051/2093] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:46:16 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:16.560439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd698cbb-35b0-4967-b905-08cf76d6df21 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:16.577496 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd698cbb-35b0-4967-b905-08cf76d6df21 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:16.686160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266ce1e-bcf2-4cae-85d0-064e33adf00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:16.688246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266ce1e-bcf2-4cae-85d0-064e33adf00d admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:16.689125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266ce1e-bcf2-4cae-85d0-064e33adf00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:16.690405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266ce1e-bcf2-4cae-85d0-064e33adf00d admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:16.692217 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1266ce1e-bcf2-4cae-85d0-064e33adf00d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:16.696511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1266ce1e-bcf2-4cae-85d0-064e33adf00d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:16.760867 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1043/2094] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:46:16 2022] POST /identity/v3/users => generated 288 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:46:16.765651 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f5138-68e3-4c9b-946d-f82fc7515716 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:16.795112 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f5138-68e3-4c9b-946d-f82fc7515716 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:16.815026 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffea1b-6894-41cc-9909-c5369dd446cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:16.816298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffea1b-6894-41cc-9909-c5369dd446cf admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:16.826502 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffea1b-6894-41cc-9909-c5369dd446cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:16.827140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffea1b-6894-41cc-9909-c5369dd446cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:16.830338 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ffea1b-6894-41cc-9909-c5369dd446cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:16.835295 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ffea1b-6894-41cc-9909-c5369dd446cf admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:16.880483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1052/2095] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:16.882448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266ce1e-bcf2-4cae-85d0-064e33adf00d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:16.917918 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266ce1e-bcf2-4cae-85d0-064e33adf00d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:16.944924 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b08505-62c2-4f13-96b0-dc0ea2879720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:16.945921 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b08505-62c2-4f13-96b0-dc0ea2879720 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:16.946359 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b08505-62c2-4f13-96b0-dc0ea2879720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:16.951392 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b08505-62c2-4f13-96b0-dc0ea2879720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:16.953199 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b08505-62c2-4f13-96b0-dc0ea2879720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:16.957172 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b08505-62c2-4f13-96b0-dc0ea2879720 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:17.000991 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1044/2096] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:17.002825 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffea1b-6894-41cc-9909-c5369dd446cf admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:17.048056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffea1b-6894-41cc-9909-c5369dd446cf admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:17.079169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb00d36-a904-45ab-b2de-91fbcad62387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:17.080290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00d36-a904-45ab-b2de-91fbcad62387 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:17.080744 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00d36-a904-45ab-b2de-91fbcad62387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:17.096333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00d36-a904-45ab-b2de-91fbcad62387 admin admin] PATH_INFO: `/v3/projects/00b7bf66c4de4faeb141ee88fd3e4099/users/38651df4c41c45869289f542aad733da/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:17.141168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb00d36-a904-45ab-b2de-91fbcad62387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00b7bf66c4de4faeb141ee88fd3e4099, user_id=38651df4c41c45869289f542aad733da, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:17.143048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb00d36-a904-45ab-b2de-91fbcad62387 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:17.216368 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1053/2097] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:17 2022] PUT /identity/v3/projects/00b7bf66c4de4faeb141ee88fd3e4099/users/38651df4c41c45869289f542aad733da/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:17.228472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b0802-4bda-403a-bb76-221f34b8ed60 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:17.229116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b0802-4bda-403a-bb76-221f34b8ed60 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:17.229251 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b0802-4bda-403a-bb76-221f34b8ed60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:17.274939 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-eb6b0802-4bda-403a-bb76-221f34b8ed60 None None] MFA Rules not processed for user `38651df4c41c45869289f542aad733da`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:17.278871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-eb6b0802-4bda-403a-bb76-221f34b8ed60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:17.295529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:17.463692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1045/2098] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:17.479381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb00d36-a904-45ab-b2de-91fbcad62387 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:17.481579 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-3eb00d36-a904-45ab-b2de-91fbcad62387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:17.599577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb00d36-a904-45ab-b2de-91fbcad62387 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:17.604103 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4f58b-b37c-432f-8f62-c17f5cf16af7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:17.604910 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4f58b-b37c-432f-8f62-c17f5cf16af7 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:17.605331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4f58b-b37c-432f-8f62-c17f5cf16af7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:17.605699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4f58b-b37c-432f-8f62-c17f5cf16af7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:17.627600 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4f58b-b37c-432f-8f62-c17f5cf16af7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:17.630422 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4f58b-b37c-432f-8f62-c17f5cf16af7 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:17.659785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1054/2099] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:21.893220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a0998-955e-46bb-91b2-f524edc8a312 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:21.894106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a0998-955e-46bb-91b2-f524edc8a312 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:21.894106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a0998-955e-46bb-91b2-f524edc8a312 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:21.895577 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1046/2100] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 26 05:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:46:21.911291 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7109da-f55e-4152-bbfa-dee9935acb21 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:21.911840 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7109da-f55e-4152-bbfa-dee9935acb21 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:21.912172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7109da-f55e-4152-bbfa-dee9935acb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:21.936568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-5a7109da-f55e-4152-bbfa-dee9935acb21 None None] MFA Rules not processed for user `67fcd65eeb994b2381b83480e030f2ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:21.938248 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5a7109da-f55e-4152-bbfa-dee9935acb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:22.028432 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a0998-955e-46bb-91b2-f524edc8a312 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:22.044775 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-505a0998-955e-46bb-91b2-f524edc8a312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:22.110342 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1055/2101] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 26 05:46:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:22.126747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7109da-f55e-4152-bbfa-dee9935acb21 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:22.129382 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5a7109da-f55e-4152-bbfa-dee9935acb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:22.146583 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7109da-f55e-4152-bbfa-dee9935acb21 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:22.152282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26892a0-7907-4011-ae07-d85ddab52912 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:22.153620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26892a0-7907-4011-ae07-d85ddab52912 service placement] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:22.153620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26892a0-7907-4011-ae07-d85ddab52912 service placement] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:22.153846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26892a0-7907-4011-ae07-d85ddab52912 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:22.164777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26892a0-7907-4011-ae07-d85ddab52912 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:22.166541 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26892a0-7907-4011-ae07-d85ddab52912 service placement] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:22.178540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1056/2102] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:22.208492 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a0998-955e-46bb-91b2-f524edc8a312 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:22.224628 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb429d-9c2e-4e6d-aad4-6cb19cb32644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:22.225774 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb429d-9c2e-4e6d-aad4-6cb19cb32644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:22.226242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb429d-9c2e-4e6d-aad4-6cb19cb32644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:22.226707 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb429d-9c2e-4e6d-aad4-6cb19cb32644 admin admin] PATH_INFO: `/v3/users/d3842855395b47a5adc10c7480b5a4ca` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:22.228653 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb429d-9c2e-4e6d-aad4-6cb19cb32644 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3842855395b47a5adc10c7480b5a4ca)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:22.231757 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb429d-9c2e-4e6d-aad4-6cb19cb32644 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:22.309584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b5cb429d-9c2e-4e6d-aad4-6cb19cb32644 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3842855395b47a5adc10c7480b5a4ca'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:22.326849 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b5cb429d-9c2e-4e6d-aad4-6cb19cb32644 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3842855395b47a5adc10c7480b5a4ca'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:22.329066 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b5cb429d-9c2e-4e6d-aad4-6cb19cb32644 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3842855395b47a5adc10c7480b5a4ca'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:22.330113 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b5cb429d-9c2e-4e6d-aad4-6cb19cb32644 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3842855395b47a5adc10c7480b5a4ca'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:22.351130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b5cb429d-9c2e-4e6d-aad4-6cb19cb32644 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3842855395b47a5adc10c7480b5a4ca'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:22.369770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1047/2103] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:22 2022] DELETE /identity/v3/users/d3842855395b47a5adc10c7480b5a4ca => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:22.643674 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26892a0-7907-4011-ae07-d85ddab52912 service placement] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:22.656018 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:22.670648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26892a0-7907-4011-ae07-d85ddab52912 service placement] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:22.743796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d93b0e-a712-4cc0-a61b-214759bb9920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:22.744861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d93b0e-a712-4cc0-a61b-214759bb9920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:22.745569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d93b0e-a712-4cc0-a61b-214759bb9920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:22.746029 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d93b0e-a712-4cc0-a61b-214759bb9920 admin admin] PATH_INFO: `/v3/projects/7edddf22ec7f41aea66bcf3e30a3d3ca` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:22.748109 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d93b0e-a712-4cc0-a61b-214759bb9920 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7edddf22ec7f41aea66bcf3e30a3d3ca)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:22.753126 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d93b0e-a712-4cc0-a61b-214759bb9920 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:22.830901 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d7d93b0e-a712-4cc0-a61b-214759bb9920 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7edddf22ec7f41aea66bcf3e30a3d3ca', 'request_id': 'req-d7d93b0e-a712-4cc0-a61b-214759bb9920'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:22.844279 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d7d93b0e-a712-4cc0-a61b-214759bb9920 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7edddf22ec7f41aea66bcf3e30a3d3ca', 'request_id': 'req-d7d93b0e-a712-4cc0-a61b-214759bb9920'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:22.854602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d7d93b0e-a712-4cc0-a61b-214759bb9920 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7edddf22ec7f41aea66bcf3e30a3d3ca', 'request_id': 'req-d7d93b0e-a712-4cc0-a61b-214759bb9920'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:22.866997 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d7d93b0e-a712-4cc0-a61b-214759bb9920 admin admin] The token cache is being invalidate because project 7edddf22ec7f41aea66bcf3e30a3d3ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:22.867292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d7d93b0e-a712-4cc0-a61b-214759bb9920 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:22.868000 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d7d93b0e-a712-4cc0-a61b-214759bb9920 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:22.869672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1057/2104] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:22 2022] DELETE /identity/v3/projects/7edddf22ec7f41aea66bcf3e30a3d3ca => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:22.874881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7527659-4e46-4b82-a2be-fc85035e5977 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:22.875241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7527659-4e46-4b82-a2be-fc85035e5977 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:22.875582 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7527659-4e46-4b82-a2be-fc85035e5977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:22.896204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-e7527659-4e46-4b82-a2be-fc85035e5977 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:22.897615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e7527659-4e46-4b82-a2be-fc85035e5977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:22.931343 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:22.981554 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1048/2105] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:22.989767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb428fd-976c-4abf-959c-cb197eac320b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:22.990343 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb428fd-976c-4abf-959c-cb197eac320b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:22.990343 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb428fd-976c-4abf-959c-cb197eac320b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:23.016514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-ccb428fd-976c-4abf-959c-cb197eac320b None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:23.018009 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ccb428fd-976c-4abf-959c-cb197eac320b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:23.026366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1058/2106] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:23.031145 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7527659-4e46-4b82-a2be-fc85035e5977 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:23.042175 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7527659-4e46-4b82-a2be-fc85035e5977 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:23.048987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea7833-1282-47f0-ad1f-72970d9a1468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:23.049661 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea7833-1282-47f0-ad1f-72970d9a1468 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:23.049940 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea7833-1282-47f0-ad1f-72970d9a1468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:23.050244 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea7833-1282-47f0-ad1f-72970d9a1468 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:23.050960 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea7833-1282-47f0-ad1f-72970d9a1468 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:46:23.051488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea7833-1282-47f0-ad1f-72970d9a1468 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:23.053169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea7833-1282-47f0-ad1f-72970d9a1468 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:23.067024 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1049/2107] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:46:23.071689 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb428fd-976c-4abf-959c-cb197eac320b None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:23.081857 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb428fd-976c-4abf-959c-cb197eac320b None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:23.089888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ace1f-8e06-4642-ba19-ac69b6356ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:23.091017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ace1f-8e06-4642-ba19-ac69b6356ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:23.091584 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ace1f-8e06-4642-ba19-ac69b6356ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:23.092108 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ace1f-8e06-4642-ba19-ac69b6356ba6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:23.093343 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ace1f-8e06-4642-ba19-ac69b6356ba6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:23.097032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ace1f-8e06-4642-ba19-ac69b6356ba6 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:23.136190 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1059/2108] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:46:23 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:23.141225 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea7833-1282-47f0-ad1f-72970d9a1468 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:23.153282 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea7833-1282-47f0-ad1f-72970d9a1468 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:23.258449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a32aa-45ef-4d88-bc50-fed29dba9c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:23.259082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a32aa-45ef-4d88-bc50-fed29dba9c57 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:23.259346 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a32aa-45ef-4d88-bc50-fed29dba9c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:23.259613 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a32aa-45ef-4d88-bc50-fed29dba9c57 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:23.260280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a32aa-45ef-4d88-bc50-fed29dba9c57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:23.262251 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a32aa-45ef-4d88-bc50-fed29dba9c57 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:23.286303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1050/2109] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:46:23 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:23.291736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ace1f-8e06-4642-ba19-ac69b6356ba6 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:23.304002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ace1f-8e06-4642-ba19-ac69b6356ba6 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:23.309918 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914d911-e621-48f7-9903-70402b4cf412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:23.310561 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914d911-e621-48f7-9903-70402b4cf412 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:23.310862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914d911-e621-48f7-9903-70402b4cf412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:23.311110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914d911-e621-48f7-9903-70402b4cf412 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:23.311772 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d914d911-e621-48f7-9903-70402b4cf412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:23.313458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d914d911-e621-48f7-9903-70402b4cf412 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:23.325416 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1060/2110] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:23.330406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a32aa-45ef-4d88-bc50-fed29dba9c57 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:23.342188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a32aa-45ef-4d88-bc50-fed29dba9c57 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:23.347707 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376e861-ae50-40a3-801f-4295545035cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:23.348340 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376e861-ae50-40a3-801f-4295545035cc admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:23.348614 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376e861-ae50-40a3-801f-4295545035cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:23.348883 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376e861-ae50-40a3-801f-4295545035cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:23.349596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376e861-ae50-40a3-801f-4295545035cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:23.351192 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376e861-ae50-40a3-801f-4295545035cc admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:23.363118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1051/2111] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:23.370477 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914d911-e621-48f7-9903-70402b4cf412 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:23.390563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914d911-e621-48f7-9903-70402b4cf412 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:23.399519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5b202-65da-4d4e-af6b-9ed75ac81e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:23.401294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5b202-65da-4d4e-af6b-9ed75ac81e01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:23.401294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5b202-65da-4d4e-af6b-9ed75ac81e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:23.401646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5b202-65da-4d4e-af6b-9ed75ac81e01 admin admin] PATH_INFO: `/v3/projects/8981a5af3bbf4feca4c40d379b3b7508/users/abb04f3e8fa64078b232ddab0b718a6d/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:23.424345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5b202-65da-4d4e-af6b-9ed75ac81e01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8981a5af3bbf4feca4c40d379b3b7508, user_id=abb04f3e8fa64078b232ddab0b718a6d, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:23.430039 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5b202-65da-4d4e-af6b-9ed75ac81e01 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:23.456225 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1061/2112] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:23 2022] PUT /identity/v3/projects/8981a5af3bbf4feca4c40d379b3b7508/users/abb04f3e8fa64078b232ddab0b718a6d/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:23.481847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473f3d2-ca31-4ccc-8c9f-e34ae1b082d1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:23.482476 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473f3d2-ca31-4ccc-8c9f-e34ae1b082d1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:23.482856 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473f3d2-ca31-4ccc-8c9f-e34ae1b082d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:23.520537 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7473f3d2-ca31-4ccc-8c9f-e34ae1b082d1 None None] MFA Rules not processed for user `abb04f3e8fa64078b232ddab0b718a6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:23.522528 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7473f3d2-ca31-4ccc-8c9f-e34ae1b082d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:23.611677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1052/2113] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:23.616647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5b202-65da-4d4e-af6b-9ed75ac81e01 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:23.656185 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:23.702421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5b202-65da-4d4e-af6b-9ed75ac81e01 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:23.713515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54040741-a6dd-498d-92fa-fdc4b16192d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:23.713515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54040741-a6dd-498d-92fa-fdc4b16192d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:23.713515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54040741-a6dd-498d-92fa-fdc4b16192d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:23.713515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54040741-a6dd-498d-92fa-fdc4b16192d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:23.714049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54040741-a6dd-498d-92fa-fdc4b16192d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:23.715109 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54040741-a6dd-498d-92fa-fdc4b16192d0 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:23.759355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1062/2114] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:46:23 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:46:23.760683 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473f3d2-ca31-4ccc-8c9f-e34ae1b082d1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:23.772785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473f3d2-ca31-4ccc-8c9f-e34ae1b082d1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:23.804315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:23.859107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a398f-a014-49ad-bea2-1af2dc5899e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:23.860291 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a398f-a014-49ad-bea2-1af2dc5899e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:23.860776 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a398f-a014-49ad-bea2-1af2dc5899e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:23.861322 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a398f-a014-49ad-bea2-1af2dc5899e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:23.862516 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a398f-a014-49ad-bea2-1af2dc5899e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:23.866320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a398f-a014-49ad-bea2-1af2dc5899e6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:23.902580 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1053/2115] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:46:23 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:23.911177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54040741-a6dd-498d-92fa-fdc4b16192d0 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:23.930728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54040741-a6dd-498d-92fa-fdc4b16192d0 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:23.940536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c8c45-9e37-46e3-a371-6cb149bc23b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:23.941564 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c8c45-9e37-46e3-a371-6cb149bc23b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:23.942141 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c8c45-9e37-46e3-a371-6cb149bc23b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:23.942558 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c8c45-9e37-46e3-a371-6cb149bc23b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:23.943776 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c8c45-9e37-46e3-a371-6cb149bc23b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:23.946836 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c8c45-9e37-46e3-a371-6cb149bc23b1 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:23.962220 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1063/2116] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:46:23.968055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a398f-a014-49ad-bea2-1af2dc5899e6 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:23.985621 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a398f-a014-49ad-bea2-1af2dc5899e6 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:23.993011 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e575c-50af-4d64-8ebc-bd1bf55ee454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:23.994206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e575c-50af-4d64-8ebc-bd1bf55ee454 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:23.994641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e575c-50af-4d64-8ebc-bd1bf55ee454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:23.995083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e575c-50af-4d64-8ebc-bd1bf55ee454 admin admin] PATH_INFO: `/v3/projects/07acf6b7f02048c7931dc69a816278df/users/64553fc2765b4c4e9fba3d9440f150ca/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:24.028647 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e575c-50af-4d64-8ebc-bd1bf55ee454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07acf6b7f02048c7931dc69a816278df, user_id=64553fc2765b4c4e9fba3d9440f150ca, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:24.034719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e575c-50af-4d64-8ebc-bd1bf55ee454 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:24.061712 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1054/2117] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:23 2022] PUT /identity/v3/projects/07acf6b7f02048c7931dc69a816278df/users/64553fc2765b4c4e9fba3d9440f150ca/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:24.065876 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c8c45-9e37-46e3-a371-6cb149bc23b1 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:24.076950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c8c45-9e37-46e3-a371-6cb149bc23b1 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:24.153625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd94460-4fa1-451d-961a-7c25c6a8a69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:24.154552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd94460-4fa1-451d-961a-7c25c6a8a69a admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:24.155021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd94460-4fa1-451d-961a-7c25c6a8a69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:24.155468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd94460-4fa1-451d-961a-7c25c6a8a69a admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:24.156795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd94460-4fa1-451d-961a-7c25c6a8a69a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:24.158843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd94460-4fa1-451d-961a-7c25c6a8a69a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:24.173806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1064/2118] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:46:24.179241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e575c-50af-4d64-8ebc-bd1bf55ee454 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:24.192730 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e575c-50af-4d64-8ebc-bd1bf55ee454 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:24.208597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c7a75-dcfb-47d3-a99e-fa2b54dd8e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:24.209517 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c7a75-dcfb-47d3-a99e-fa2b54dd8e34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:24.209882 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c7a75-dcfb-47d3-a99e-fa2b54dd8e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:24.210194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c7a75-dcfb-47d3-a99e-fa2b54dd8e34 admin admin] PATH_INFO: `/v3/projects/07acf6b7f02048c7931dc69a816278df/users/64553fc2765b4c4e9fba3d9440f150ca/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:24.213716 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2c7a75-dcfb-47d3-a99e-fa2b54dd8e34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07acf6b7f02048c7931dc69a816278df, user_id=64553fc2765b4c4e9fba3d9440f150ca, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:24.216718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2c7a75-dcfb-47d3-a99e-fa2b54dd8e34 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:24.233666 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1055/2119] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:24 2022] PUT /identity/v3/projects/07acf6b7f02048c7931dc69a816278df/users/64553fc2765b4c4e9fba3d9440f150ca/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:46:24.252248 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e85300-03f5-4981-9747-d25570e910ea None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:24.252798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e85300-03f5-4981-9747-d25570e910ea None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:24.253345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e85300-03f5-4981-9747-d25570e910ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:24.295176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-06e85300-03f5-4981-9747-d25570e910ea None None] MFA Rules not processed for user `64553fc2765b4c4e9fba3d9440f150ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:24.296887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-06e85300-03f5-4981-9747-d25570e910ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:24.455079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1065/2120] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:24.466651 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c7a75-dcfb-47d3-a99e-fa2b54dd8e34 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:24.469894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0c2c7a75-dcfb-47d3-a99e-fa2b54dd8e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:24.564762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c7a75-dcfb-47d3-a99e-fa2b54dd8e34 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:24.571586 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ad18a-7026-4d28-a0de-d0773620f364 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:24.572461 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ad18a-7026-4d28-a0de-d0773620f364 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:24.572914 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ad18a-7026-4d28-a0de-d0773620f364 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:24.573261 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ad18a-7026-4d28-a0de-d0773620f364 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:24.587413 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ad18a-7026-4d28-a0de-d0773620f364 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:24.588866 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ad18a-7026-4d28-a0de-d0773620f364 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:24.599459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1056/2121] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:25.089697 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e85300-03f5-4981-9747-d25570e910ea None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:25.123352 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:25.166007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e85300-03f5-4981-9747-d25570e910ea None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:25.171641 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f7c92-9cda-4f11-bc5a-a8c805e243be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:25.172900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f7c92-9cda-4f11-bc5a-a8c805e243be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:25.173072 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f7c92-9cda-4f11-bc5a-a8c805e243be admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:25.173407 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f7c92-9cda-4f11-bc5a-a8c805e243be admin admin] PATH_INFO: `/v3/users/abb04f3e8fa64078b232ddab0b718a6d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:25.175075 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f7c92-9cda-4f11-bc5a-a8c805e243be admin admin] RBAC: Authorizing `identity:delete_user(user_id=abb04f3e8fa64078b232ddab0b718a6d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:25.177336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f7c92-9cda-4f11-bc5a-a8c805e243be admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:25.273844 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-661f7c92-9cda-4f11-bc5a-a8c805e243be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abb04f3e8fa64078b232ddab0b718a6d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:25.301959 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-661f7c92-9cda-4f11-bc5a-a8c805e243be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abb04f3e8fa64078b232ddab0b718a6d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:25.302619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-661f7c92-9cda-4f11-bc5a-a8c805e243be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abb04f3e8fa64078b232ddab0b718a6d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:25.304542 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-661f7c92-9cda-4f11-bc5a-a8c805e243be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abb04f3e8fa64078b232ddab0b718a6d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:25.312649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-661f7c92-9cda-4f11-bc5a-a8c805e243be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abb04f3e8fa64078b232ddab0b718a6d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:25.341114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1066/2122] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:25 2022] DELETE /identity/v3/users/abb04f3e8fa64078b232ddab0b718a6d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:25.349817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981bcad-4b95-4fba-8e52-ce26dfa844b2 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:25.350437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981bcad-4b95-4fba-8e52-ce26dfa844b2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:25.350525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981bcad-4b95-4fba-8e52-ce26dfa844b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:25.377562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b981bcad-4b95-4fba-8e52-ce26dfa844b2 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:25.378920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b981bcad-4b95-4fba-8e52-ce26dfa844b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:25.457259 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1057/2123] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:25.473783 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f7c92-9cda-4f11-bc5a-a8c805e243be admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:25.476064 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-661f7c92-9cda-4f11-bc5a-a8c805e243be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:25.563719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f7c92-9cda-4f11-bc5a-a8c805e243be admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:25.569642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd0ba2-1d18-445a-ada6-4b3cdf625f57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:25.570730 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd0ba2-1d18-445a-ada6-4b3cdf625f57 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:25.571228 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd0ba2-1d18-445a-ada6-4b3cdf625f57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:25.571613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd0ba2-1d18-445a-ada6-4b3cdf625f57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:25.583443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd0ba2-1d18-445a-ada6-4b3cdf625f57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:25.584907 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd0ba2-1d18-445a-ada6-4b3cdf625f57 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:25.597355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1067/2124] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:25.988448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981bcad-4b95-4fba-8e52-ce26dfa844b2 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:25.990414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b981bcad-4b95-4fba-8e52-ce26dfa844b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:26.011969 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981bcad-4b95-4fba-8e52-ce26dfa844b2 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:26.024583 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8db13-9520-4ce8-b855-d66d3e12d93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:26.025897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8db13-9520-4ce8-b855-d66d3e12d93d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:26.026231 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8db13-9520-4ce8-b855-d66d3e12d93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:26.026509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8db13-9520-4ce8-b855-d66d3e12d93d admin admin] PATH_INFO: `/v3/projects/8981a5af3bbf4feca4c40d379b3b7508` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:26.028073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8db13-9520-4ce8-b855-d66d3e12d93d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8981a5af3bbf4feca4c40d379b3b7508)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:26.030140 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8db13-9520-4ce8-b855-d66d3e12d93d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:26.051213 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:26.115337 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-49a8db13-9520-4ce8-b855-d66d3e12d93d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8981a5af3bbf4feca4c40d379b3b7508', 'request_id': 'req-49a8db13-9520-4ce8-b855-d66d3e12d93d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:26.125565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-49a8db13-9520-4ce8-b855-d66d3e12d93d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8981a5af3bbf4feca4c40d379b3b7508', 'request_id': 'req-49a8db13-9520-4ce8-b855-d66d3e12d93d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:26.139769 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-49a8db13-9520-4ce8-b855-d66d3e12d93d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8981a5af3bbf4feca4c40d379b3b7508', 'request_id': 'req-49a8db13-9520-4ce8-b855-d66d3e12d93d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:26.156402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-49a8db13-9520-4ce8-b855-d66d3e12d93d admin admin] The token cache is being invalidate because project 8981a5af3bbf4feca4c40d379b3b7508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:26.156697 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-49a8db13-9520-4ce8-b855-d66d3e12d93d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:26.157471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-49a8db13-9520-4ce8-b855-d66d3e12d93d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:26.159270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1058/2125] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:25 2022] DELETE /identity/v3/projects/8981a5af3bbf4feca4c40d379b3b7508 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:26.163454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd0ba2-1d18-445a-ada6-4b3cdf625f57 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:26.173607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd0ba2-1d18-445a-ada6-4b3cdf625f57 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:26.241411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:26.259034 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a6e27-c00c-4c2c-a45f-6b31685dd4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:26.259676 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a6e27-c00c-4c2c-a45f-6b31685dd4a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:26.259955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a6e27-c00c-4c2c-a45f-6b31685dd4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:26.260230 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a6e27-c00c-4c2c-a45f-6b31685dd4a7 admin admin] PATH_INFO: `/v3/users/64553fc2765b4c4e9fba3d9440f150ca` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:26.261414 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a6e27-c00c-4c2c-a45f-6b31685dd4a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64553fc2765b4c4e9fba3d9440f150ca)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:26.263351 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a6e27-c00c-4c2c-a45f-6b31685dd4a7 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:26.315981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-345a6e27-c00c-4c2c-a45f-6b31685dd4a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64553fc2765b4c4e9fba3d9440f150ca'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:26.326086 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-345a6e27-c00c-4c2c-a45f-6b31685dd4a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64553fc2765b4c4e9fba3d9440f150ca'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:26.326798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-345a6e27-c00c-4c2c-a45f-6b31685dd4a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64553fc2765b4c4e9fba3d9440f150ca'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:26.327645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-345a6e27-c00c-4c2c-a45f-6b31685dd4a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64553fc2765b4c4e9fba3d9440f150ca'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:26.336477 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-345a6e27-c00c-4c2c-a45f-6b31685dd4a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64553fc2765b4c4e9fba3d9440f150ca'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:26.360819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1068/2126] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:26 2022] DELETE /identity/v3/users/64553fc2765b4c4e9fba3d9440f150ca => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:46:26.817173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8db13-9520-4ce8-b855-d66d3e12d93d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:26.826187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-49a8db13-9520-4ce8-b855-d66d3e12d93d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:26.843698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8db13-9520-4ce8-b855-d66d3e12d93d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:26.926699 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd88c4-5a84-4b2e-8ffd-8679cd56fac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:26.927796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd88c4-5a84-4b2e-8ffd-8679cd56fac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:26.928234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd88c4-5a84-4b2e-8ffd-8679cd56fac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:26.928890 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd88c4-5a84-4b2e-8ffd-8679cd56fac0 admin admin] PATH_INFO: `/v3/projects/07acf6b7f02048c7931dc69a816278df` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:26.930575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd88c4-5a84-4b2e-8ffd-8679cd56fac0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07acf6b7f02048c7931dc69a816278df)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:26.932720 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd88c4-5a84-4b2e-8ffd-8679cd56fac0 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:27.029267 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-10dd88c4-5a84-4b2e-8ffd-8679cd56fac0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07acf6b7f02048c7931dc69a816278df', 'request_id': 'req-10dd88c4-5a84-4b2e-8ffd-8679cd56fac0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:27.041263 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-10dd88c4-5a84-4b2e-8ffd-8679cd56fac0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07acf6b7f02048c7931dc69a816278df', 'request_id': 'req-10dd88c4-5a84-4b2e-8ffd-8679cd56fac0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:27.053802 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-10dd88c4-5a84-4b2e-8ffd-8679cd56fac0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07acf6b7f02048c7931dc69a816278df', 'request_id': 'req-10dd88c4-5a84-4b2e-8ffd-8679cd56fac0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:27.065828 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-10dd88c4-5a84-4b2e-8ffd-8679cd56fac0 admin admin] The token cache is being invalidate because project 07acf6b7f02048c7931dc69a816278df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:27.066177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-10dd88c4-5a84-4b2e-8ffd-8679cd56fac0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:27.066826 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-10dd88c4-5a84-4b2e-8ffd-8679cd56fac0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:27.068342 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1059/2127] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:26 2022] DELETE /identity/v3/projects/07acf6b7f02048c7931dc69a816278df => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:27.073905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bd5dc-e5e0-4a0e-874e-1275cb673284 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:27.074240 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bd5dc-e5e0-4a0e-874e-1275cb673284 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:27.074470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bd5dc-e5e0-4a0e-874e-1275cb673284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:27.101336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-6e2bd5dc-e5e0-4a0e-874e-1275cb673284 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:27.102573 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6e2bd5dc-e5e0-4a0e-874e-1275cb673284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:27.180986 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1069/2128] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:27.189443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ed475-ffd9-4c23-9040-53e4d991c531 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:27.189975 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ed475-ffd9-4c23-9040-53e4d991c531 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:27.190316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ed475-ffd9-4c23-9040-53e4d991c531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:27.212854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-b78ed475-ffd9-4c23-9040-53e4d991c531 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:27.214239 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b78ed475-ffd9-4c23-9040-53e4d991c531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:27.232608 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1060/2129] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:27.237901 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bd5dc-e5e0-4a0e-874e-1275cb673284 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:27.253525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bd5dc-e5e0-4a0e-874e-1275cb673284 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:27.260014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1498fd-a564-4561-b061-369a57634d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:27.260785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1498fd-a564-4561-b061-369a57634d86 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:27.261814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1498fd-a564-4561-b061-369a57634d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:27.261814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1498fd-a564-4561-b061-369a57634d86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:27.262713 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1498fd-a564-4561-b061-369a57634d86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:46:27.263176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1498fd-a564-4561-b061-369a57634d86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:27.265604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1498fd-a564-4561-b061-369a57634d86 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:27.281994 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1070/2130] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:46:27.286578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ed475-ffd9-4c23-9040-53e4d991c531 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:27.296681 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ed475-ffd9-4c23-9040-53e4d991c531 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:27.303455 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0449a25-31ff-42ff-815a-705c5b6021d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:27.304464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0449a25-31ff-42ff-815a-705c5b6021d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:27.304945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0449a25-31ff-42ff-815a-705c5b6021d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:27.305444 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0449a25-31ff-42ff-815a-705c5b6021d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:27.306249 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0449a25-31ff-42ff-815a-705c5b6021d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:27.309512 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0449a25-31ff-42ff-815a-705c5b6021d5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:27.356101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1061/2131] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:46:27 2022] POST /identity/v3/projects => generated 381 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:46:27.360219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1498fd-a564-4561-b061-369a57634d86 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:27.371844 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1498fd-a564-4561-b061-369a57634d86 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:27.438406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:27.454690 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5cbdf-8337-4b64-8a7e-0c089c9ec531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:27.455808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5cbdf-8337-4b64-8a7e-0c089c9ec531 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:27.456070 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5cbdf-8337-4b64-8a7e-0c089c9ec531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:27.456338 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5cbdf-8337-4b64-8a7e-0c089c9ec531 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:27.457027 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5cbdf-8337-4b64-8a7e-0c089c9ec531 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:27.458990 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5cbdf-8337-4b64-8a7e-0c089c9ec531 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:27.490751 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1071/2132] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:46:27 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:27.495362 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0449a25-31ff-42ff-815a-705c5b6021d5 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:27.506501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0449a25-31ff-42ff-815a-705c5b6021d5 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:27.511114 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61433ebd-0db2-43a1-b15c-ccc652c541b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:27.511724 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61433ebd-0db2-43a1-b15c-ccc652c541b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:27.511996 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61433ebd-0db2-43a1-b15c-ccc652c541b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:27.512260 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61433ebd-0db2-43a1-b15c-ccc652c541b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:27.513009 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61433ebd-0db2-43a1-b15c-ccc652c541b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:27.514656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61433ebd-0db2-43a1-b15c-ccc652c541b6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:27.516191 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:27.531782 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1062/2133] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:27.531989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5cbdf-8337-4b64-8a7e-0c089c9ec531 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:27.544480 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5cbdf-8337-4b64-8a7e-0c089c9ec531 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:27.549839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ca89b-3cad-48c6-b218-bead3b28c188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:27.550647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ca89b-3cad-48c6-b218-bead3b28c188 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:27.550845 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ca89b-3cad-48c6-b218-bead3b28c188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:27.551118 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ca89b-3cad-48c6-b218-bead3b28c188 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:27.551771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ca89b-3cad-48c6-b218-bead3b28c188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:27.553415 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ca89b-3cad-48c6-b218-bead3b28c188 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:27.564838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1072/2134] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:27.569484 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61433ebd-0db2-43a1-b15c-ccc652c541b6 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:27.580345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61433ebd-0db2-43a1-b15c-ccc652c541b6 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:27.585224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda0d71-764d-430d-8465-bd33065e8e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:27.588226 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0d71-764d-430d-8465-bd33065e8e0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:27.588607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0d71-764d-430d-8465-bd33065e8e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:27.588955 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0d71-764d-430d-8465-bd33065e8e0b admin admin] PATH_INFO: `/v3/projects/da2ed663c2e849469114475767a93d69/users/1e0a99f5dca24851a9d80335a7c799e1/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:27.605099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda0d71-764d-430d-8465-bd33065e8e0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=da2ed663c2e849469114475767a93d69, user_id=1e0a99f5dca24851a9d80335a7c799e1, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:27.607901 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda0d71-764d-430d-8465-bd33065e8e0b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:27.622727 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1063/2135] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:27 2022] PUT /identity/v3/projects/da2ed663c2e849469114475767a93d69/users/1e0a99f5dca24851a9d80335a7c799e1/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:27.634231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18490c6-2fb2-464d-9571-ea28b5379528 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:27.634646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18490c6-2fb2-464d-9571-ea28b5379528 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:27.635024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18490c6-2fb2-464d-9571-ea28b5379528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:27.667761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d18490c6-2fb2-464d-9571-ea28b5379528 None None] MFA Rules not processed for user `1e0a99f5dca24851a9d80335a7c799e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:27.669060 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d18490c6-2fb2-464d-9571-ea28b5379528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:27.749459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1073/2136] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:27.754082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda0d71-764d-430d-8465-bd33065e8e0b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:27.840633 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda0d71-764d-430d-8465-bd33065e8e0b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:27.850531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62be77-a8d0-4746-a1c9-6747bd213a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:27.851602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62be77-a8d0-4746-a1c9-6747bd213a6b admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:27.852086 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62be77-a8d0-4746-a1c9-6747bd213a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:27.852622 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62be77-a8d0-4746-a1c9-6747bd213a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:27.856266 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62be77-a8d0-4746-a1c9-6747bd213a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:27.857776 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62be77-a8d0-4746-a1c9-6747bd213a6b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:27.897202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1064/2137] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:46:27 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:27.902334 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18490c6-2fb2-464d-9571-ea28b5379528 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:27.933619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18490c6-2fb2-464d-9571-ea28b5379528 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:28.081980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a346cef-b2c6-4d37-b1ee-38c1c7cf9c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:28.083237 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a346cef-b2c6-4d37-b1ee-38c1c7cf9c58 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:28.083736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a346cef-b2c6-4d37-b1ee-38c1c7cf9c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:28.084418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a346cef-b2c6-4d37-b1ee-38c1c7cf9c58 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:28.085293 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a346cef-b2c6-4d37-b1ee-38c1c7cf9c58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:28.087641 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a346cef-b2c6-4d37-b1ee-38c1c7cf9c58 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:28.122912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1074/2138] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:46:27 2022] POST /identity/v3/users => generated 298 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:28.128017 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62be77-a8d0-4746-a1c9-6747bd213a6b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:28.148599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62be77-a8d0-4746-a1c9-6747bd213a6b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:28.158745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb2b1b-87d6-46a9-bd2f-5d519c26c42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:28.159661 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb2b1b-87d6-46a9-bd2f-5d519c26c42c admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:28.160097 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb2b1b-87d6-46a9-bd2f-5d519c26c42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:28.160495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb2b1b-87d6-46a9-bd2f-5d519c26c42c admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:28.162617 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb2b1b-87d6-46a9-bd2f-5d519c26c42c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:28.164842 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb2b1b-87d6-46a9-bd2f-5d519c26c42c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:28.183736 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1065/2139] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:28.190482 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a346cef-b2c6-4d37-b1ee-38c1c7cf9c58 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:28.205442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a346cef-b2c6-4d37-b1ee-38c1c7cf9c58 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:28.221506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97452a-6e6e-46ce-ac05-2b33d739dc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:28.222299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97452a-6e6e-46ce-ac05-2b33d739dc63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:28.222850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97452a-6e6e-46ce-ac05-2b33d739dc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:28.223199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97452a-6e6e-46ce-ac05-2b33d739dc63 admin admin] PATH_INFO: `/v3/projects/196a8bb9c1bd4d709aeb48bfa44b0862/users/aedbb81f7c5343259aceff794d328ff1/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:28.241619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97452a-6e6e-46ce-ac05-2b33d739dc63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=196a8bb9c1bd4d709aeb48bfa44b0862, user_id=aedbb81f7c5343259aceff794d328ff1, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:28.244442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97452a-6e6e-46ce-ac05-2b33d739dc63 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:28.259752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1075/2140] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:28 2022] PUT /identity/v3/projects/196a8bb9c1bd4d709aeb48bfa44b0862/users/aedbb81f7c5343259aceff794d328ff1/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:28.263872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb2b1b-87d6-46a9-bd2f-5d519c26c42c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:28.275487 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb2b1b-87d6-46a9-bd2f-5d519c26c42c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:28.335567 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa405b40-de23-4ad8-9400-9985b4c4fd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:28.336593 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa405b40-de23-4ad8-9400-9985b4c4fd94 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:28.336891 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa405b40-de23-4ad8-9400-9985b4c4fd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:28.337184 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa405b40-de23-4ad8-9400-9985b4c4fd94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:28.337843 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa405b40-de23-4ad8-9400-9985b4c4fd94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:28.339541 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa405b40-de23-4ad8-9400-9985b4c4fd94 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:28.350374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1066/2141] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:28.354946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97452a-6e6e-46ce-ac05-2b33d739dc63 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:28.366268 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97452a-6e6e-46ce-ac05-2b33d739dc63 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:28.372493 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad347f-4ae7-4492-b34d-8fd3b4a0ee94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:28.373256 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad347f-4ae7-4492-b34d-8fd3b4a0ee94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:28.373532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad347f-4ae7-4492-b34d-8fd3b4a0ee94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:28.373806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad347f-4ae7-4492-b34d-8fd3b4a0ee94 admin admin] PATH_INFO: `/v3/projects/196a8bb9c1bd4d709aeb48bfa44b0862/users/aedbb81f7c5343259aceff794d328ff1/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:28.375940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad347f-4ae7-4492-b34d-8fd3b4a0ee94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=196a8bb9c1bd4d709aeb48bfa44b0862, user_id=aedbb81f7c5343259aceff794d328ff1, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:28.378629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad347f-4ae7-4492-b34d-8fd3b4a0ee94 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:28.390978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1076/2142] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:28 2022] PUT /identity/v3/projects/196a8bb9c1bd4d709aeb48bfa44b0862/users/aedbb81f7c5343259aceff794d328ff1/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:28.402641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d8f6b-8232-4db7-8d2f-d9592ff4bcd5 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:28.402641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d8f6b-8232-4db7-8d2f-d9592ff4bcd5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:28.402825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d8f6b-8232-4db7-8d2f-d9592ff4bcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:28.444141 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-975d8f6b-8232-4db7-8d2f-d9592ff4bcd5 None None] MFA Rules not processed for user `aedbb81f7c5343259aceff794d328ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:28.445769 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-975d8f6b-8232-4db7-8d2f-d9592ff4bcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:28.498045 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:28.569193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1067/2143] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:28.580599 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad347f-4ae7-4492-b34d-8fd3b4a0ee94 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:28.582607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-e6ad347f-4ae7-4492-b34d-8fd3b4a0ee94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:28.604135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:28.664123 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad347f-4ae7-4492-b34d-8fd3b4a0ee94 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:28.668484 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c0a8a-c53f-42b2-83f0-efb7ec5082e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:28.669248 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0a8a-c53f-42b2-83f0-efb7ec5082e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:28.669582 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0a8a-c53f-42b2-83f0-efb7ec5082e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:28.670261 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0a8a-c53f-42b2-83f0-efb7ec5082e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:28.683139 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c0a8a-c53f-42b2-83f0-efb7ec5082e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:28.684582 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c0a8a-c53f-42b2-83f0-efb7ec5082e8 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:28.695838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1077/2144] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:28.838754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d8f6b-8232-4db7-8d2f-d9592ff4bcd5 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:28.850558 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d8f6b-8232-4db7-8d2f-d9592ff4bcd5 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:28.855139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d0546-3d0f-4793-bfc1-fdf37d0dc9d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:28.858547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d0546-3d0f-4793-bfc1-fdf37d0dc9d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:28.858547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d0546-3d0f-4793-bfc1-fdf37d0dc9d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:28.858547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d0546-3d0f-4793-bfc1-fdf37d0dc9d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:28.940293 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d0546-3d0f-4793-bfc1-fdf37d0dc9d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:28.942760 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d0546-3d0f-4793-bfc1-fdf37d0dc9d2 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:28.956937 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1068/2145] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:32.735780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c0a8a-c53f-42b2-83f0-efb7ec5082e8 service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:32.737346 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-596c0a8a-c53f-42b2-83f0-efb7ec5082e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:32.813945 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c0a8a-c53f-42b2-83f0-efb7ec5082e8 service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:32.818759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efab9b-d66d-48d7-91e0-648f5d8ccaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:32.819401 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efab9b-d66d-48d7-91e0-648f5d8ccaa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:32.819670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efab9b-d66d-48d7-91e0-648f5d8ccaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:32.819937 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efab9b-d66d-48d7-91e0-648f5d8ccaa4 admin admin] PATH_INFO: `/v3/users/38651df4c41c45869289f542aad733da` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:32.821087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efab9b-d66d-48d7-91e0-648f5d8ccaa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38651df4c41c45869289f542aad733da)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:32.823066 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efab9b-d66d-48d7-91e0-648f5d8ccaa4 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:32.888682 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-75efab9b-d66d-48d7-91e0-648f5d8ccaa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38651df4c41c45869289f542aad733da'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:32.897655 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-75efab9b-d66d-48d7-91e0-648f5d8ccaa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38651df4c41c45869289f542aad733da'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:32.898370 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-75efab9b-d66d-48d7-91e0-648f5d8ccaa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38651df4c41c45869289f542aad733da'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:32.899036 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-75efab9b-d66d-48d7-91e0-648f5d8ccaa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38651df4c41c45869289f542aad733da'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:32.908297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-75efab9b-d66d-48d7-91e0-648f5d8ccaa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38651df4c41c45869289f542aad733da'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:32.932306 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1078/2146] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:32 2022] DELETE /identity/v3/users/38651df4c41c45869289f542aad733da => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:32.937573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f22797-cacb-46e2-9426-06b876ef44ad None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:32.938469 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f22797-cacb-46e2-9426-06b876ef44ad None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:32.938738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f22797-cacb-46e2-9426-06b876ef44ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:32.971638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-e0f22797-cacb-46e2-9426-06b876ef44ad None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:32.973046 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e0f22797-cacb-46e2-9426-06b876ef44ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:32.989488 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efab9b-d66d-48d7-91e0-648f5d8ccaa4 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:32.991117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-75efab9b-d66d-48d7-91e0-648f5d8ccaa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:33.046296 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:33.083457 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1069/2147] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:33.100301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f22797-cacb-46e2-9426-06b876ef44ad None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:33.110935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e0f22797-cacb-46e2-9426-06b876ef44ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:33.111227 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efab9b-d66d-48d7-91e0-648f5d8ccaa4 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:33.117719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe03dba-7493-4139-a3b4-946236ae8598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:33.118465 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe03dba-7493-4139-a3b4-946236ae8598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:33.118745 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe03dba-7493-4139-a3b4-946236ae8598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:33.119238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe03dba-7493-4139-a3b4-946236ae8598 admin admin] PATH_INFO: `/v3/users/1e0a99f5dca24851a9d80335a7c799e1` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:33.120547 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe03dba-7493-4139-a3b4-946236ae8598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e0a99f5dca24851a9d80335a7c799e1)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:33.123142 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe03dba-7493-4139-a3b4-946236ae8598 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:33.204123 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f22797-cacb-46e2-9426-06b876ef44ad None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:33.209624 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f8289-8727-42c2-9f2e-49d2a4699979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:33.209966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f8289-8727-42c2-9f2e-49d2a4699979 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:33.210356 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f8289-8727-42c2-9f2e-49d2a4699979 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:33.210748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f8289-8727-42c2-9f2e-49d2a4699979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:33.227568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-dbe03dba-7493-4139-a3b4-946236ae8598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e0a99f5dca24851a9d80335a7c799e1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:33.238934 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f8289-8727-42c2-9f2e-49d2a4699979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:33.241177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f8289-8727-42c2-9f2e-49d2a4699979 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:33.252002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-dbe03dba-7493-4139-a3b4-946236ae8598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e0a99f5dca24851a9d80335a7c799e1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:33.253217 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-dbe03dba-7493-4139-a3b4-946236ae8598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e0a99f5dca24851a9d80335a7c799e1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:33.256148 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-dbe03dba-7493-4139-a3b4-946236ae8598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e0a99f5dca24851a9d80335a7c799e1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:33.260030 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1070/2148] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:33.278040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-dbe03dba-7493-4139-a3b4-946236ae8598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e0a99f5dca24851a9d80335a7c799e1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:33.318738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1079/2149] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:32 2022] DELETE /identity/v3/users/1e0a99f5dca24851a9d80335a7c799e1 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:33.323808 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd692278-5f52-4686-b21e-1d06e55a0595 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:33.324337 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd692278-5f52-4686-b21e-1d06e55a0595 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:33.325514 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd692278-5f52-4686-b21e-1d06e55a0595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:33.362994 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-cd692278-5f52-4686-b21e-1d06e55a0595 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:33.364347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cd692278-5f52-4686-b21e-1d06e55a0595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:33.369495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:33.455942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1071/2150] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:33.466794 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe03dba-7493-4139-a3b4-946236ae8598 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:33.469947 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-dbe03dba-7493-4139-a3b4-946236ae8598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:33.550181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe03dba-7493-4139-a3b4-946236ae8598 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:33.554273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec7b30-7bc1-426e-a97b-03ad9a2c0cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:33.555452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec7b30-7bc1-426e-a97b-03ad9a2c0cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:33.555614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec7b30-7bc1-426e-a97b-03ad9a2c0cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:33.555770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec7b30-7bc1-426e-a97b-03ad9a2c0cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:33.580905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec7b30-7bc1-426e-a97b-03ad9a2c0cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:33.583702 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec7b30-7bc1-426e-a97b-03ad9a2c0cf0 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:33.600110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1080/2151] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:33.718341 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd692278-5f52-4686-b21e-1d06e55a0595 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:33.720655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cd692278-5f52-4686-b21e-1d06e55a0595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:33.743232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd692278-5f52-4686-b21e-1d06e55a0595 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:33.751282 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccdb84-f426-4487-9668-b55a1c8d5525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:33.752214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccdb84-f426-4487-9668-b55a1c8d5525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:33.752427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccdb84-f426-4487-9668-b55a1c8d5525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:33.753104 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccdb84-f426-4487-9668-b55a1c8d5525 admin admin] PATH_INFO: `/v3/projects/00b7bf66c4de4faeb141ee88fd3e4099` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:33.754440 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dccdb84-f426-4487-9668-b55a1c8d5525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00b7bf66c4de4faeb141ee88fd3e4099)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:33.757105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dccdb84-f426-4487-9668-b55a1c8d5525 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:33.819271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3dccdb84-f426-4487-9668-b55a1c8d5525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00b7bf66c4de4faeb141ee88fd3e4099', 'request_id': 'req-3dccdb84-f426-4487-9668-b55a1c8d5525'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:33.827305 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3dccdb84-f426-4487-9668-b55a1c8d5525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00b7bf66c4de4faeb141ee88fd3e4099', 'request_id': 'req-3dccdb84-f426-4487-9668-b55a1c8d5525'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:33.846895 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3dccdb84-f426-4487-9668-b55a1c8d5525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00b7bf66c4de4faeb141ee88fd3e4099', 'request_id': 'req-3dccdb84-f426-4487-9668-b55a1c8d5525'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:33.861367 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3dccdb84-f426-4487-9668-b55a1c8d5525 admin admin] The token cache is being invalidate because project 00b7bf66c4de4faeb141ee88fd3e4099 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:33.861672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3dccdb84-f426-4487-9668-b55a1c8d5525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:33.862478 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3dccdb84-f426-4487-9668-b55a1c8d5525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:33.864007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1072/2152] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:33 2022] DELETE /identity/v3/projects/00b7bf66c4de4faeb141ee88fd3e4099 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:33.993823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec7b30-7bc1-426e-a97b-03ad9a2c0cf0 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:33.996697 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-b4ec7b30-7bc1-426e-a97b-03ad9a2c0cf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:34.029942 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec7b30-7bc1-426e-a97b-03ad9a2c0cf0 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:34.102281 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:34.110507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f447e2c-7605-4563-a254-bd7424fc5afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:34.111260 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f447e2c-7605-4563-a254-bd7424fc5afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:34.111768 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f447e2c-7605-4563-a254-bd7424fc5afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:34.112099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f447e2c-7605-4563-a254-bd7424fc5afa admin admin] PATH_INFO: `/v3/projects/da2ed663c2e849469114475767a93d69` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:34.114516 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f447e2c-7605-4563-a254-bd7424fc5afa admin admin] RBAC: Authorizing `identity:delete_project(project_id=da2ed663c2e849469114475767a93d69)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:34.116651 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f447e2c-7605-4563-a254-bd7424fc5afa admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:34.188393 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7f447e2c-7605-4563-a254-bd7424fc5afa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da2ed663c2e849469114475767a93d69', 'request_id': 'req-7f447e2c-7605-4563-a254-bd7424fc5afa'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:34.197756 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7f447e2c-7605-4563-a254-bd7424fc5afa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da2ed663c2e849469114475767a93d69', 'request_id': 'req-7f447e2c-7605-4563-a254-bd7424fc5afa'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:34.210980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7f447e2c-7605-4563-a254-bd7424fc5afa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da2ed663c2e849469114475767a93d69', 'request_id': 'req-7f447e2c-7605-4563-a254-bd7424fc5afa'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:34.239111 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7f447e2c-7605-4563-a254-bd7424fc5afa admin admin] The token cache is being invalidate because project da2ed663c2e849469114475767a93d69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:34.239111 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7f447e2c-7605-4563-a254-bd7424fc5afa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:34.239111 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7f447e2c-7605-4563-a254-bd7424fc5afa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:34.240039 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1081/2153] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:33 2022] DELETE /identity/v3/projects/da2ed663c2e849469114475767a93d69 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:46:34.244302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccdb84-f426-4487-9668-b55a1c8d5525 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:34.245844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3dccdb84-f426-4487-9668-b55a1c8d5525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:34.259347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccdb84-f426-4487-9668-b55a1c8d5525 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:34.325220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e781b50-4fa8-46ab-aa32-8e5fc180403b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:34.325951 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e781b50-4fa8-46ab-aa32-8e5fc180403b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:34.326298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e781b50-4fa8-46ab-aa32-8e5fc180403b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:34.326633 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e781b50-4fa8-46ab-aa32-8e5fc180403b admin admin] PATH_INFO: `/v3/users/aedbb81f7c5343259aceff794d328ff1` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:34.327877 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e781b50-4fa8-46ab-aa32-8e5fc180403b admin admin] RBAC: Authorizing `identity:delete_user(user_id=aedbb81f7c5343259aceff794d328ff1)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:34.329908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e781b50-4fa8-46ab-aa32-8e5fc180403b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:34.372747 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:34.386344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6e781b50-4fa8-46ab-aa32-8e5fc180403b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aedbb81f7c5343259aceff794d328ff1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:34.394730 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6e781b50-4fa8-46ab-aa32-8e5fc180403b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aedbb81f7c5343259aceff794d328ff1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:34.395673 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6e781b50-4fa8-46ab-aa32-8e5fc180403b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aedbb81f7c5343259aceff794d328ff1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:34.397013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6e781b50-4fa8-46ab-aa32-8e5fc180403b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aedbb81f7c5343259aceff794d328ff1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:34.423006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6e781b50-4fa8-46ab-aa32-8e5fc180403b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aedbb81f7c5343259aceff794d328ff1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:34.439135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1073/2154] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:34 2022] DELETE /identity/v3/users/aedbb81f7c5343259aceff794d328ff1 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:34.892991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f447e2c-7605-4563-a254-bd7424fc5afa admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:34.894790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7f447e2c-7605-4563-a254-bd7424fc5afa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:34.912300 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f447e2c-7605-4563-a254-bd7424fc5afa admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:34.984748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206aa274-a2ba-4433-b0e4-ede99b0b7ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:34.985491 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aa274-a2ba-4433-b0e4-ede99b0b7ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:34.985822 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aa274-a2ba-4433-b0e4-ede99b0b7ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:34.986270 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aa274-a2ba-4433-b0e4-ede99b0b7ee1 admin admin] PATH_INFO: `/v3/projects/196a8bb9c1bd4d709aeb48bfa44b0862` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:34.987795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206aa274-a2ba-4433-b0e4-ede99b0b7ee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=196a8bb9c1bd4d709aeb48bfa44b0862)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:34.990164 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206aa274-a2ba-4433-b0e4-ede99b0b7ee1 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:35.042907 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-206aa274-a2ba-4433-b0e4-ede99b0b7ee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '196a8bb9c1bd4d709aeb48bfa44b0862', 'request_id': 'req-206aa274-a2ba-4433-b0e4-ede99b0b7ee1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:35.049512 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-206aa274-a2ba-4433-b0e4-ede99b0b7ee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '196a8bb9c1bd4d709aeb48bfa44b0862', 'request_id': 'req-206aa274-a2ba-4433-b0e4-ede99b0b7ee1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:35.058415 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-206aa274-a2ba-4433-b0e4-ede99b0b7ee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '196a8bb9c1bd4d709aeb48bfa44b0862', 'request_id': 'req-206aa274-a2ba-4433-b0e4-ede99b0b7ee1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:35.071195 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-206aa274-a2ba-4433-b0e4-ede99b0b7ee1 admin admin] The token cache is being invalidate because project 196a8bb9c1bd4d709aeb48bfa44b0862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:35.071609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-206aa274-a2ba-4433-b0e4-ede99b0b7ee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:35.072817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-206aa274-a2ba-4433-b0e4-ede99b0b7ee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:35.074018 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1082/2155] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:34 2022] DELETE /identity/v3/projects/196a8bb9c1bd4d709aeb48bfa44b0862 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:35.079370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12ec20-eb05-4497-8329-50daebc2aa7c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:35.079659 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12ec20-eb05-4497-8329-50daebc2aa7c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:35.079927 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12ec20-eb05-4497-8329-50daebc2aa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:35.102582 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-0c12ec20-eb05-4497-8329-50daebc2aa7c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:35.103800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0c12ec20-eb05-4497-8329-50daebc2aa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:35.168311 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1074/2156] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:35.179800 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e209a-51f7-4419-9720-2ba895b5491c None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:35.180047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e209a-51f7-4419-9720-2ba895b5491c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:35.180382 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e209a-51f7-4419-9720-2ba895b5491c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:35.207378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-0e6e209a-51f7-4419-9720-2ba895b5491c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:35.208793 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-0e6e209a-51f7-4419-9720-2ba895b5491c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:35.219971 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1083/2157] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:35.224501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12ec20-eb05-4497-8329-50daebc2aa7c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:35.235513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12ec20-eb05-4497-8329-50daebc2aa7c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:35.241416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b3b19-3c41-436f-97ca-2afee4630f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:35.242050 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b3b19-3c41-436f-97ca-2afee4630f0b admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:35.242326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b3b19-3c41-436f-97ca-2afee4630f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:35.242597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b3b19-3c41-436f-97ca-2afee4630f0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:35.243258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b3b19-3c41-436f-97ca-2afee4630f0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:46:35.243560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b3b19-3c41-436f-97ca-2afee4630f0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:35.245240 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b3b19-3c41-436f-97ca-2afee4630f0b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:35.259035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1075/2158] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:46:35.264101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e209a-51f7-4419-9720-2ba895b5491c None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:35.274660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e209a-51f7-4419-9720-2ba895b5491c None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:35.280327 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a8010-cf53-4597-a817-9301c73e5378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:35.280961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a8010-cf53-4597-a817-9301c73e5378 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:35.281272 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a8010-cf53-4597-a817-9301c73e5378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:35.281542 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a8010-cf53-4597-a817-9301c73e5378 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:35.282192 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a8010-cf53-4597-a817-9301c73e5378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:35.284133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a8010-cf53-4597-a817-9301c73e5378 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:35.294671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:35.316866 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1084/2159] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:46:35 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:35.321854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b3b19-3c41-436f-97ca-2afee4630f0b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:35.332977 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b3b19-3c41-436f-97ca-2afee4630f0b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:35.413623 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7861cd3-42e6-4d37-a8d6-633a6fc6f916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:35.414275 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7861cd3-42e6-4d37-a8d6-633a6fc6f916 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:35.414601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7861cd3-42e6-4d37-a8d6-633a6fc6f916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:35.414823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7861cd3-42e6-4d37-a8d6-633a6fc6f916 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:35.415563 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7861cd3-42e6-4d37-a8d6-633a6fc6f916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:35.417653 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7861cd3-42e6-4d37-a8d6-633a6fc6f916 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:35.444547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1076/2160] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:46:35 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:35.449529 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a8010-cf53-4597-a817-9301c73e5378 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:35.461866 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a8010-cf53-4597-a817-9301c73e5378 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:35.467771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd2f46-7ec6-48a1-828c-597df17c51a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:35.468621 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd2f46-7ec6-48a1-828c-597df17c51a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:35.468745 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd2f46-7ec6-48a1-828c-597df17c51a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:35.469018 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd2f46-7ec6-48a1-828c-597df17c51a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:35.469723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd2f46-7ec6-48a1-828c-597df17c51a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:35.471339 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd2f46-7ec6-48a1-828c-597df17c51a1 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:35.483245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1085/2161] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:46:35.487103 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7861cd3-42e6-4d37-a8d6-633a6fc6f916 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:35.497425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7861cd3-42e6-4d37-a8d6-633a6fc6f916 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:35.502845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339e1c3-d76d-4ce2-9c84-619cedb19096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:35.503832 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339e1c3-d76d-4ce2-9c84-619cedb19096 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:35.504117 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339e1c3-d76d-4ce2-9c84-619cedb19096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:35.504400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339e1c3-d76d-4ce2-9c84-619cedb19096 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:35.505389 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339e1c3-d76d-4ce2-9c84-619cedb19096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:35.507084 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339e1c3-d76d-4ce2-9c84-619cedb19096 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:35.519803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1077/2162] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:35.524612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd2f46-7ec6-48a1-828c-597df17c51a1 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:35.540268 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd2f46-7ec6-48a1-828c-597df17c51a1 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:35.546052 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d226b-db15-4de6-9963-ffd1b0485cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:35.546629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d226b-db15-4de6-9963-ffd1b0485cf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:35.546896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d226b-db15-4de6-9963-ffd1b0485cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:35.547192 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d226b-db15-4de6-9963-ffd1b0485cf1 admin admin] PATH_INFO: `/v3/projects/d739de121005470e857548efe4b26572/users/1c9732ff61c94a31bdf516169c67a00b/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:35.563397 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d226b-db15-4de6-9963-ffd1b0485cf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d739de121005470e857548efe4b26572, user_id=1c9732ff61c94a31bdf516169c67a00b, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:35.566076 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d226b-db15-4de6-9963-ffd1b0485cf1 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:35.581410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1086/2163] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:35 2022] PUT /identity/v3/projects/d739de121005470e857548efe4b26572/users/1c9732ff61c94a31bdf516169c67a00b/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:35.594618 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a786c-75c7-4405-b3ee-06ca05a58e8e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:35.595018 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a786c-75c7-4405-b3ee-06ca05a58e8e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:35.595335 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a786c-75c7-4405-b3ee-06ca05a58e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:35.625615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-bd1a786c-75c7-4405-b3ee-06ca05a58e8e None None] MFA Rules not processed for user `1c9732ff61c94a31bdf516169c67a00b`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:35.626840 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bd1a786c-75c7-4405-b3ee-06ca05a58e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:35.632224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:35.720336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1078/2164] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:35.725603 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d226b-db15-4de6-9963-ffd1b0485cf1 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:35.802203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d226b-db15-4de6-9963-ffd1b0485cf1 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:35.809602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d3070-0244-49b9-af84-bedd5d1c834a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:35.810235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d3070-0244-49b9-af84-bedd5d1c834a admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:35.810511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d3070-0244-49b9-af84-bedd5d1c834a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:35.810778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d3070-0244-49b9-af84-bedd5d1c834a admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:35.811432 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d3070-0244-49b9-af84-bedd5d1c834a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:35.813438 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d3070-0244-49b9-af84-bedd5d1c834a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:35.845309 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1087/2165] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:46:35 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:35.850936 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a786c-75c7-4405-b3ee-06ca05a58e8e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:35.862631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a786c-75c7-4405-b3ee-06ca05a58e8e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:35.930830 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe73f8b-15df-4ba1-aab3-e18818a3d3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:35.931453 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe73f8b-15df-4ba1-aab3-e18818a3d3ad admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:35.931731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe73f8b-15df-4ba1-aab3-e18818a3d3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:35.932001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe73f8b-15df-4ba1-aab3-e18818a3d3ad admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:35.932645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe73f8b-15df-4ba1-aab3-e18818a3d3ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:35.934663 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe73f8b-15df-4ba1-aab3-e18818a3d3ad admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:35.956574 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1079/2166] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:46:35 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:35.962884 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d3070-0244-49b9-af84-bedd5d1c834a admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:35.976622 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d3070-0244-49b9-af84-bedd5d1c834a admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:35.982812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d902a17-9943-43d8-ad6e-59d5b31d442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:35.983425 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d902a17-9943-43d8-ad6e-59d5b31d442b admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:35.983703 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d902a17-9943-43d8-ad6e-59d5b31d442b admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:35.983950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d902a17-9943-43d8-ad6e-59d5b31d442b admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:35.984769 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d902a17-9943-43d8-ad6e-59d5b31d442b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:35.986397 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d902a17-9943-43d8-ad6e-59d5b31d442b admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:35.998470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1088/2167] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:36.005024 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe73f8b-15df-4ba1-aab3-e18818a3d3ad admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:36.017632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe73f8b-15df-4ba1-aab3-e18818a3d3ad admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:36.023895 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3869e-6c36-450f-adc8-06933b0d476b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:36.024598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3869e-6c36-450f-adc8-06933b0d476b admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:36.024878 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3869e-6c36-450f-adc8-06933b0d476b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:36.025179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3869e-6c36-450f-adc8-06933b0d476b admin admin] PATH_INFO: `/v3/projects/c279fcd284ca45aa8c19f83245be0779/users/8d133c33029c4759a068760dc7491cba/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:36.042249 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3869e-6c36-450f-adc8-06933b0d476b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c279fcd284ca45aa8c19f83245be0779, user_id=8d133c33029c4759a068760dc7491cba, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:36.045102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3869e-6c36-450f-adc8-06933b0d476b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:36.064672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1080/2168] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:36 2022] PUT /identity/v3/projects/c279fcd284ca45aa8c19f83245be0779/users/8d133c33029c4759a068760dc7491cba/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:36.068977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d902a17-9943-43d8-ad6e-59d5b31d442b admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:36.079459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d902a17-9943-43d8-ad6e-59d5b31d442b admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:36.139376 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38a7dd-52cd-47c7-81f0-4a2341e2a93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:36.141479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38a7dd-52cd-47c7-81f0-4a2341e2a93c admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:36.141479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38a7dd-52cd-47c7-81f0-4a2341e2a93c admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:36.141479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38a7dd-52cd-47c7-81f0-4a2341e2a93c admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:36.141479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38a7dd-52cd-47c7-81f0-4a2341e2a93c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:36.144778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38a7dd-52cd-47c7-81f0-4a2341e2a93c admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:36.153812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1089/2169] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:36.158391 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3869e-6c36-450f-adc8-06933b0d476b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:36.169267 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3869e-6c36-450f-adc8-06933b0d476b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:36.174606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afcef7-9353-4cd4-afb5-304352479b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:36.175280 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afcef7-9353-4cd4-afb5-304352479b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:36.175606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afcef7-9353-4cd4-afb5-304352479b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:36.175900 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afcef7-9353-4cd4-afb5-304352479b06 admin admin] PATH_INFO: `/v3/projects/c279fcd284ca45aa8c19f83245be0779/users/8d133c33029c4759a068760dc7491cba/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:36.177816 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afcef7-9353-4cd4-afb5-304352479b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c279fcd284ca45aa8c19f83245be0779, user_id=8d133c33029c4759a068760dc7491cba, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:36.180505 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afcef7-9353-4cd4-afb5-304352479b06 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:36.191886 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1081/2170] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:36 2022] PUT /identity/v3/projects/c279fcd284ca45aa8c19f83245be0779/users/8d133c33029c4759a068760dc7491cba/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:36.202675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888ed09-91b9-4baf-8cad-ffe95f28bc29 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:36.202965 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888ed09-91b9-4baf-8cad-ffe95f28bc29 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:36.203242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888ed09-91b9-4baf-8cad-ffe95f28bc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:36.247862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-6888ed09-91b9-4baf-8cad-ffe95f28bc29 None None] MFA Rules not processed for user `8d133c33029c4759a068760dc7491cba`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:36.249715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6888ed09-91b9-4baf-8cad-ffe95f28bc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:36.267780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:36.354285 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1090/2171] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:36.367402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afcef7-9353-4cd4-afb5-304352479b06 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:36.369465 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-03afcef7-9353-4cd4-afb5-304352479b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:36.433885 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afcef7-9353-4cd4-afb5-304352479b06 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:36.438138 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124786c6-5393-40d3-bf39-d43e685eda34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:36.438770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124786c6-5393-40d3-bf39-d43e685eda34 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:36.439092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124786c6-5393-40d3-bf39-d43e685eda34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:36.439367 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124786c6-5393-40d3-bf39-d43e685eda34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:36.454236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:36.524297 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124786c6-5393-40d3-bf39-d43e685eda34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:36.527127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124786c6-5393-40d3-bf39-d43e685eda34 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:36.544051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1082/2172] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:41.462566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bac261-bd54-4af4-a16f-75dea6000ef2 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:41.463324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bac261-bd54-4af4-a16f-75dea6000ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:41.463324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bac261-bd54-4af4-a16f-75dea6000ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:41.485939 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-33bac261-bd54-4af4-a16f-75dea6000ef2 None None] MFA Rules not processed for user `04c7152003c4491abea8305c26f366ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:41.487324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-33bac261-bd54-4af4-a16f-75dea6000ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:41.567563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1091/2173] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:44.400041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124786c6-5393-40d3-bf39-d43e685eda34 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:44.401812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-124786c6-5393-40d3-bf39-d43e685eda34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:44.417808 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124786c6-5393-40d3-bf39-d43e685eda34 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:44.423073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbe02a-5180-4df0-ad92-1134cb8935fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:44.423787 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbe02a-5180-4df0-ad92-1134cb8935fa service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:44.424116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbe02a-5180-4df0-ad92-1134cb8935fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:44.424480 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbe02a-5180-4df0-ad92-1134cb8935fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:44.435799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbe02a-5180-4df0-ad92-1134cb8935fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:44.437350 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbe02a-5180-4df0-ad92-1134cb8935fa service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:44.448237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1083/2174] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:46:45.719592 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660513f-72e2-4065-a23e-270dc3ccdfb8 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:45.720451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660513f-72e2-4065-a23e-270dc3ccdfb8 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:45.720451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660513f-72e2-4065-a23e-270dc3ccdfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:45.748697 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-4660513f-72e2-4065-a23e-270dc3ccdfb8 None None] MFA Rules not processed for user `0216bf67389e4627bf953b7dcc97a88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:45.749989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4660513f-72e2-4065-a23e-270dc3ccdfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:45.830345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1092/2175] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 26 05:46:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:45.886726 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbe02a-5180-4df0-ad92-1134cb8935fa service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:45.888361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-39cbe02a-5180-4df0-ad92-1134cb8935fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:45.975626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbe02a-5180-4df0-ad92-1134cb8935fa service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:45.981630 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6a80f-492c-4127-877c-7134c7c1b98f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f258aba84d4dd59ebe8de973ac65c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:45.982368 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6a80f-492c-4127-877c-7134c7c1b98f service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:45.982720 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6a80f-492c-4127-877c-7134c7c1b98f service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:45.983448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6a80f-492c-4127-877c-7134c7c1b98f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:45.994551 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6a80f-492c-4127-877c-7134c7c1b98f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:45.996463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6a80f-492c-4127-877c-7134c7c1b98f service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:46.006960 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1084/2176] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 26 05:46:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:46:46.893047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef012c-ae2f-43a8-b528-adbd9b367587 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:46.893721 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef012c-ae2f-43a8-b528-adbd9b367587 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:46.893721 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef012c-ae2f-43a8-b528-adbd9b367587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:46.921424 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-45ef012c-ae2f-43a8-b528-adbd9b367587 None None] MFA Rules not processed for user `0216bf67389e4627bf953b7dcc97a88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:46.922897 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-45ef012c-ae2f-43a8-b528-adbd9b367587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:46.931875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1093/2177] 10.222.0.57 () {58 vars in 967 bytes} [Mon Sep 26 05:46:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:46.951512 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6a80f-492c-4127-877c-7134c7c1b98f service swift] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:46.966849 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6a80f-492c-4127-877c-7134c7c1b98f service swift] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:46.973384 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05af6e-efe9-49b4-a747-682a45cd2023 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f258aba84d4dd59ebe8de973ac65c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:46.974144 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05af6e-efe9-49b4-a747-682a45cd2023 service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:46.974522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05af6e-efe9-49b4-a747-682a45cd2023 service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:46.974912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05af6e-efe9-49b4-a747-682a45cd2023 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:46.985061 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05af6e-efe9-49b4-a747-682a45cd2023 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:46.986770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05af6e-efe9-49b4-a747-682a45cd2023 service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:47.005830 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1085/2178] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 26 05:46:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:46:47.421098 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4caad-ca74-4cb5-aaaa-811473ad2467 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:47.421518 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4caad-ca74-4cb5-aaaa-811473ad2467 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:47.421827 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4caad-ca74-4cb5-aaaa-811473ad2467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:47.445199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-9fa4caad-ca74-4cb5-aaaa-811473ad2467 None None] MFA Rules not processed for user `0216bf67389e4627bf953b7dcc97a88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:47.446634 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9fa4caad-ca74-4cb5-aaaa-811473ad2467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:47.454745 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1094/2179] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 26 05:46:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:47.470327 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05af6e-efe9-49b4-a747-682a45cd2023 service swift] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:47.502502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05af6e-efe9-49b4-a747-682a45cd2023 service swift] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:47.513021 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cbd65-56ee-4211-9bdb-3d0148421c76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f258aba84d4dd59ebe8de973ac65c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:47.514153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cbd65-56ee-4211-9bdb-3d0148421c76 service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:47.514642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cbd65-56ee-4211-9bdb-3d0148421c76 service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:47.515122 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cbd65-56ee-4211-9bdb-3d0148421c76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:47.530422 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cbd65-56ee-4211-9bdb-3d0148421c76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:47.533381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cbd65-56ee-4211-9bdb-3d0148421c76 service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:47.546316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1086/2180] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 26 05:46:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 26 05:46:48.089232 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe311f4-8c76-4713-a720-8f042eb91b9a None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:48.089846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe311f4-8c76-4713-a720-8f042eb91b9a None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:48.090106 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe311f4-8c76-4713-a720-8f042eb91b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:48.124135 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-fbe311f4-8c76-4713-a720-8f042eb91b9a None None] MFA Rules not processed for user `0216bf67389e4627bf953b7dcc97a88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:48.126240 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-fbe311f4-8c76-4713-a720-8f042eb91b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:48.139045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1095/2181] 10.222.0.57 () {58 vars in 967 bytes} [Mon Sep 26 05:46:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:48.160431 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cbd65-56ee-4211-9bdb-3d0148421c76 service swift] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:48.177459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cbd65-56ee-4211-9bdb-3d0148421c76 service swift] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:48.184141 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a7f12-5ff1-4ea7-a406-0a01d2f42ba1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f258aba84d4dd59ebe8de973ac65c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:48.185255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a7f12-5ff1-4ea7-a406-0a01d2f42ba1 service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:48.185912 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a7f12-5ff1-4ea7-a406-0a01d2f42ba1 service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:48.186388 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a7f12-5ff1-4ea7-a406-0a01d2f42ba1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:48.201710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a7f12-5ff1-4ea7-a406-0a01d2f42ba1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:48.204498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a7f12-5ff1-4ea7-a406-0a01d2f42ba1 service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:48.218225 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1087/2182] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 26 05:46:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:46:50.557632 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29baf1-465d-4068-bac8-29da9c55133c None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:50.558354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29baf1-465d-4068-bac8-29da9c55133c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:50.558354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29baf1-465d-4068-bac8-29da9c55133c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:50.582406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-1a29baf1-465d-4068-bac8-29da9c55133c None None] MFA Rules not processed for user `342ce12282264d40b2f2bb6f1fc6ba25`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:50.584136 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-1a29baf1-465d-4068-bac8-29da9c55133c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:50.642824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:50.679805 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1096/2183] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:50.846371 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a7f12-5ff1-4ea7-a406-0a01d2f42ba1 service swift] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:50.850390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9e2a7f12-5ff1-4ea7-a406-0a01d2f42ba1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:50.875210 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29baf1-465d-4068-bac8-29da9c55133c None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:50.976994 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:51.006732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a7f12-5ff1-4ea7-a406-0a01d2f42ba1 service swift] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:51.012148 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f685d7-9e67-426b-9565-60eb0ee12827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:51.013128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f685d7-9e67-426b-9565-60eb0ee12827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:51.013594 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f685d7-9e67-426b-9565-60eb0ee12827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:51.014460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f685d7-9e67-426b-9565-60eb0ee12827 admin admin] PATH_INFO: `/v3/users/97028c1b3fe141dea0fa530a4b8aff30` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:51.016253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f685d7-9e67-426b-9565-60eb0ee12827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97028c1b3fe141dea0fa530a4b8aff30)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:51.018417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f685d7-9e67-426b-9565-60eb0ee12827 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:51.031042 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29baf1-465d-4068-bac8-29da9c55133c None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:51.038880 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e66b1-afbd-4015-8a59-1ab939c0b847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:51.040377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e66b1-afbd-4015-8a59-1ab939c0b847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:51.040959 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e66b1-afbd-4015-8a59-1ab939c0b847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:51.041499 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e66b1-afbd-4015-8a59-1ab939c0b847 admin admin] PATH_INFO: `/v3/users/1c9732ff61c94a31bdf516169c67a00b` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:51.043655 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e66b1-afbd-4015-8a59-1ab939c0b847 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c9732ff61c94a31bdf516169c67a00b)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:51.047988 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e66b1-afbd-4015-8a59-1ab939c0b847 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:51.142942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-23f685d7-9e67-426b-9565-60eb0ee12827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97028c1b3fe141dea0fa530a4b8aff30'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:51.158126 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-23f685d7-9e67-426b-9565-60eb0ee12827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97028c1b3fe141dea0fa530a4b8aff30'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:51.158939 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-23f685d7-9e67-426b-9565-60eb0ee12827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97028c1b3fe141dea0fa530a4b8aff30'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:51.159582 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-23f685d7-9e67-426b-9565-60eb0ee12827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97028c1b3fe141dea0fa530a4b8aff30'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:51.169445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a89e66b1-afbd-4015-8a59-1ab939c0b847 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c9732ff61c94a31bdf516169c67a00b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:51.177286 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-23f685d7-9e67-426b-9565-60eb0ee12827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97028c1b3fe141dea0fa530a4b8aff30'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:51.185961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a89e66b1-afbd-4015-8a59-1ab939c0b847 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c9732ff61c94a31bdf516169c67a00b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:51.187396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a89e66b1-afbd-4015-8a59-1ab939c0b847 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c9732ff61c94a31bdf516169c67a00b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:51.188750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a89e66b1-afbd-4015-8a59-1ab939c0b847 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c9732ff61c94a31bdf516169c67a00b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:51.193895 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1088/2184] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:50 2022] DELETE /identity/v3/users/97028c1b3fe141dea0fa530a4b8aff30 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:51.200263 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902e13c-1428-4854-9275-c1045ca7ea23 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:51.200592 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902e13c-1428-4854-9275-c1045ca7ea23 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:51.200801 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902e13c-1428-4854-9275-c1045ca7ea23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:51.205909 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a89e66b1-afbd-4015-8a59-1ab939c0b847 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c9732ff61c94a31bdf516169c67a00b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:51.232440 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7902e13c-1428-4854-9275-c1045ca7ea23 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:51.234058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7902e13c-1428-4854-9275-c1045ca7ea23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:51.253633 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1097/2185] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:50 2022] DELETE /identity/v3/users/1c9732ff61c94a31bdf516169c67a00b => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:51.261237 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97071739-366f-4997-a754-e9a6d0f1eeac None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:51.261544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97071739-366f-4997-a754-e9a6d0f1eeac None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:51.261836 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97071739-366f-4997-a754-e9a6d0f1eeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:51.295441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-97071739-366f-4997-a754-e9a6d0f1eeac None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:51.297414 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-97071739-366f-4997-a754-e9a6d0f1eeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:51.329768 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1089/2186] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:51.344408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902e13c-1428-4854-9275-c1045ca7ea23 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:51.346594 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7902e13c-1428-4854-9275-c1045ca7ea23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:51.417865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1098/2187] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:51.433029 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97071739-366f-4997-a754-e9a6d0f1eeac None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:51.435933 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-97071739-366f-4997-a754-e9a6d0f1eeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:51.446044 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902e13c-1428-4854-9275-c1045ca7ea23 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:51.448032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:51.486329 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:51.563300 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97071739-366f-4997-a754-e9a6d0f1eeac None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:51.564822 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b08548-1ea1-4c05-aa33-7ff6adc420d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:51.566232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b08548-1ea1-4c05-aa33-7ff6adc420d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:51.566896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b08548-1ea1-4c05-aa33-7ff6adc420d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:51.567585 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b08548-1ea1-4c05-aa33-7ff6adc420d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:51.570455 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1bd94-a3cf-47e5-aa29-180345aa4bdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:51.571494 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1bd94-a3cf-47e5-aa29-180345aa4bdb service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:51.571859 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1bd94-a3cf-47e5-aa29-180345aa4bdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:51.572274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1bd94-a3cf-47e5-aa29-180345aa4bdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:51.588670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b08548-1ea1-4c05-aa33-7ff6adc420d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:51.594022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1bd94-a3cf-47e5-aa29-180345aa4bdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:51.594553 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b08548-1ea1-4c05-aa33-7ff6adc420d5 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:51.597882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1bd94-a3cf-47e5-aa29-180345aa4bdb service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:51.617824 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1090/2188] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:46:51.623848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1099/2189] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:46:52.068999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b08548-1ea1-4c05-aa33-7ff6adc420d5 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:52.072367 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-38b08548-1ea1-4c05-aa33-7ff6adc420d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:52.101373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b08548-1ea1-4c05-aa33-7ff6adc420d5 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:52.110373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2f1fd-6aec-4478-9918-564fbcdf8720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:52.111489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2f1fd-6aec-4478-9918-564fbcdf8720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:52.111985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2f1fd-6aec-4478-9918-564fbcdf8720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:52.112461 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2f1fd-6aec-4478-9918-564fbcdf8720 admin admin] PATH_INFO: `/v3/projects/9c05c9127854481aa2ff754f336e7895` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:52.114796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2f1fd-6aec-4478-9918-564fbcdf8720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c05c9127854481aa2ff754f336e7895)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:52.118972 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2f1fd-6aec-4478-9918-564fbcdf8720 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:52.240530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bbb2f1fd-6aec-4478-9918-564fbcdf8720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c05c9127854481aa2ff754f336e7895', 'request_id': 'req-bbb2f1fd-6aec-4478-9918-564fbcdf8720'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:52.260395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bbb2f1fd-6aec-4478-9918-564fbcdf8720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c05c9127854481aa2ff754f336e7895', 'request_id': 'req-bbb2f1fd-6aec-4478-9918-564fbcdf8720'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:52.283086 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bbb2f1fd-6aec-4478-9918-564fbcdf8720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c05c9127854481aa2ff754f336e7895', 'request_id': 'req-bbb2f1fd-6aec-4478-9918-564fbcdf8720'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:52.312348 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bbb2f1fd-6aec-4478-9918-564fbcdf8720 admin admin] The token cache is being invalidate because project 9c05c9127854481aa2ff754f336e7895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:52.313260 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bbb2f1fd-6aec-4478-9918-564fbcdf8720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:52.315279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bbb2f1fd-6aec-4478-9918-564fbcdf8720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:52.322187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1091/2190] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:52 2022] DELETE /identity/v3/projects/9c05c9127854481aa2ff754f336e7895 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:52.862641 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1bd94-a3cf-47e5-aa29-180345aa4bdb service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:52.865995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c8c1bd94-a3cf-47e5-aa29-180345aa4bdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:52.915299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1bd94-a3cf-47e5-aa29-180345aa4bdb service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:53.023640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:53.024435 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:53.024812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:53.025216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d admin admin] PATH_INFO: `/v3/projects/d739de121005470e857548efe4b26572` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:53.026604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d739de121005470e857548efe4b26572)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:53.028818 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:53.097725 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd739de121005470e857548efe4b26572', 'request_id': 'req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:53.106241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd739de121005470e857548efe4b26572', 'request_id': 'req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:53.117020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd739de121005470e857548efe4b26572', 'request_id': 'req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:53.133840 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d admin admin] The token cache is being invalidate because project d739de121005470e857548efe4b26572 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:53.134147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:53.135228 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:53.137028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1100/2191] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:52 2022] DELETE /identity/v3/projects/d739de121005470e857548efe4b26572 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:53.143970 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2f1fd-6aec-4478-9918-564fbcdf8720 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:53.146488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bbb2f1fd-6aec-4478-9918-564fbcdf8720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:53.169670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2f1fd-6aec-4478-9918-564fbcdf8720 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:53.290985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c062a7-8e74-4369-a1e9-3802ccad6922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:53.292089 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c062a7-8e74-4369-a1e9-3802ccad6922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:53.292532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c062a7-8e74-4369-a1e9-3802ccad6922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:53.293006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c062a7-8e74-4369-a1e9-3802ccad6922 admin admin] PATH_INFO: `/v3/users/8d133c33029c4759a068760dc7491cba` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:53.295170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c062a7-8e74-4369-a1e9-3802ccad6922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d133c33029c4759a068760dc7491cba)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:53.298998 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c062a7-8e74-4369-a1e9-3802ccad6922 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:53.364482 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e3c062a7-8e74-4369-a1e9-3802ccad6922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d133c33029c4759a068760dc7491cba'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:53.374979 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e3c062a7-8e74-4369-a1e9-3802ccad6922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d133c33029c4759a068760dc7491cba'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:53.376039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e3c062a7-8e74-4369-a1e9-3802ccad6922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d133c33029c4759a068760dc7491cba'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:53.377102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e3c062a7-8e74-4369-a1e9-3802ccad6922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d133c33029c4759a068760dc7491cba'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:53.379414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:53.395569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e3c062a7-8e74-4369-a1e9-3802ccad6922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d133c33029c4759a068760dc7491cba'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:53.410006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1092/2192] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:46:53 2022] DELETE /identity/v3/users/8d133c33029c4759a068760dc7491cba => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:46:54.037244 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:54.040185 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:54.069282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a0f32-0fb1-42f7-97e3-e6348f29944d admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:54.161567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:54.210867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50314d7f-353d-4a44-a67c-1432432147ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:54.212314 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50314d7f-353d-4a44-a67c-1432432147ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:54.212868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50314d7f-353d-4a44-a67c-1432432147ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:54.213417 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50314d7f-353d-4a44-a67c-1432432147ec admin admin] PATH_INFO: `/v3/projects/c279fcd284ca45aa8c19f83245be0779` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:54.221069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50314d7f-353d-4a44-a67c-1432432147ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=c279fcd284ca45aa8c19f83245be0779)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:54.229158 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50314d7f-353d-4a44-a67c-1432432147ec admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:54.313739 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-50314d7f-353d-4a44-a67c-1432432147ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c279fcd284ca45aa8c19f83245be0779', 'request_id': 'req-50314d7f-353d-4a44-a67c-1432432147ec'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:54.325958 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-50314d7f-353d-4a44-a67c-1432432147ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c279fcd284ca45aa8c19f83245be0779', 'request_id': 'req-50314d7f-353d-4a44-a67c-1432432147ec'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:54.342834 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-50314d7f-353d-4a44-a67c-1432432147ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c279fcd284ca45aa8c19f83245be0779', 'request_id': 'req-50314d7f-353d-4a44-a67c-1432432147ec'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:54.359357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-50314d7f-353d-4a44-a67c-1432432147ec admin admin] The token cache is being invalidate because project c279fcd284ca45aa8c19f83245be0779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:46:54.359735 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-50314d7f-353d-4a44-a67c-1432432147ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:54.360601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-50314d7f-353d-4a44-a67c-1432432147ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:46:54.362610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1101/2193] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:46:54 2022] DELETE /identity/v3/projects/c279fcd284ca45aa8c19f83245be0779 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:54.369571 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81fb05-69ec-47da-85ce-ab5f8db5c991 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:54.370442 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81fb05-69ec-47da-85ce-ab5f8db5c991 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:54.370442 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81fb05-69ec-47da-85ce-ab5f8db5c991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:54.394838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-2a81fb05-69ec-47da-85ce-ab5f8db5c991 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:54.396183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-2a81fb05-69ec-47da-85ce-ab5f8db5c991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:54.490059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1093/2194] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:54.502347 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494191da-2ac7-4c1e-90a3-ce40bfc35671 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:54.503059 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494191da-2ac7-4c1e-90a3-ce40bfc35671 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:54.503643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494191da-2ac7-4c1e-90a3-ce40bfc35671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:54.534220 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-494191da-2ac7-4c1e-90a3-ce40bfc35671 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:54.535739 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-494191da-2ac7-4c1e-90a3-ce40bfc35671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:54.547028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1102/2195] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:46:54.554153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81fb05-69ec-47da-85ce-ab5f8db5c991 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:54.572119 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81fb05-69ec-47da-85ce-ab5f8db5c991 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:54.581754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b99040-c3ab-49f9-a465-98f64c3c7439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:54.581754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b99040-c3ab-49f9-a465-98f64c3c7439 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:54.582276 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b99040-c3ab-49f9-a465-98f64c3c7439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:54.582650 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b99040-c3ab-49f9-a465-98f64c3c7439 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:54.583880 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b99040-c3ab-49f9-a465-98f64c3c7439 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:46:54.584462 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b99040-c3ab-49f9-a465-98f64c3c7439 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:54.587997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b99040-c3ab-49f9-a465-98f64c3c7439 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:54.613719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1094/2196] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:46:54.615490 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494191da-2ac7-4c1e-90a3-ce40bfc35671 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:54.628523 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494191da-2ac7-4c1e-90a3-ce40bfc35671 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:54.636759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278c7ef-07bf-4c7b-a0cc-bf61bbbe58fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:54.637598 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278c7ef-07bf-4c7b-a0cc-bf61bbbe58fa admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:54.637971 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278c7ef-07bf-4c7b-a0cc-bf61bbbe58fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:54.638362 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278c7ef-07bf-4c7b-a0cc-bf61bbbe58fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:54.639144 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278c7ef-07bf-4c7b-a0cc-bf61bbbe58fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:54.641238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278c7ef-07bf-4c7b-a0cc-bf61bbbe58fa admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:54.677720 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1103/2197] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:46:54 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:54.682800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b99040-c3ab-49f9-a465-98f64c3c7439 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:54.693566 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b99040-c3ab-49f9-a465-98f64c3c7439 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:54.771116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bf00e-1b06-4d98-b033-88d456f1f691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:54.772162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bf00e-1b06-4d98-b033-88d456f1f691 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:54.772547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bf00e-1b06-4d98-b033-88d456f1f691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:54.772926 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bf00e-1b06-4d98-b033-88d456f1f691 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:54.773721 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328bf00e-1b06-4d98-b033-88d456f1f691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:54.776369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328bf00e-1b06-4d98-b033-88d456f1f691 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:54.802133 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1095/2198] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:46:54 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:46:54.807918 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278c7ef-07bf-4c7b-a0cc-bf61bbbe58fa admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:54.822295 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278c7ef-07bf-4c7b-a0cc-bf61bbbe58fa admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:54.830308 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39554f-336c-4700-b1ac-0c668bf7ce93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:54.831324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39554f-336c-4700-b1ac-0c668bf7ce93 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:54.831813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39554f-336c-4700-b1ac-0c668bf7ce93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:54.832306 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39554f-336c-4700-b1ac-0c668bf7ce93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:54.833618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39554f-336c-4700-b1ac-0c668bf7ce93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:54.836615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39554f-336c-4700-b1ac-0c668bf7ce93 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:54.856104 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1104/2199] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:54.863413 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bf00e-1b06-4d98-b033-88d456f1f691 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:54.881622 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bf00e-1b06-4d98-b033-88d456f1f691 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:54.890618 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19760821-a2d6-4776-bbe0-d2871d514f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:54.891487 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19760821-a2d6-4776-bbe0-d2871d514f25 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:54.891871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19760821-a2d6-4776-bbe0-d2871d514f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:54.892345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19760821-a2d6-4776-bbe0-d2871d514f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:54.893500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19760821-a2d6-4776-bbe0-d2871d514f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:54.895941 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19760821-a2d6-4776-bbe0-d2871d514f25 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:54.915343 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1096/2200] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:46:54.920457 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39554f-336c-4700-b1ac-0c668bf7ce93 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:54.931709 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39554f-336c-4700-b1ac-0c668bf7ce93 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:54.939551 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c418e12-7231-4932-89ba-b111a5703f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:54.941447 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c418e12-7231-4932-89ba-b111a5703f7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:54.942038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c418e12-7231-4932-89ba-b111a5703f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:54.942601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c418e12-7231-4932-89ba-b111a5703f7a admin admin] PATH_INFO: `/v3/projects/895bbbd2094d4d47b52753223a6db939/users/54ef0b156bcf4e408eab746b7d9aa8ec/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:54.963121 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c418e12-7231-4932-89ba-b111a5703f7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=895bbbd2094d4d47b52753223a6db939, user_id=54ef0b156bcf4e408eab746b7d9aa8ec, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:54.966683 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c418e12-7231-4932-89ba-b111a5703f7a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:54.984128 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1105/2201] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:46:54 2022] PUT /identity/v3/projects/895bbbd2094d4d47b52753223a6db939/users/54ef0b156bcf4e408eab746b7d9aa8ec/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:46:54.996523 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cdb1a-ae7e-4352-9c0b-3f73460afd84 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:54.996917 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cdb1a-ae7e-4352-9c0b-3f73460afd84 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:54.997303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cdb1a-ae7e-4352-9c0b-3f73460afd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:55.037263 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-237cdb1a-ae7e-4352-9c0b-3f73460afd84 None None] MFA Rules not processed for user `54ef0b156bcf4e408eab746b7d9aa8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:46:55.038626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-237cdb1a-ae7e-4352-9c0b-3f73460afd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:55.094026 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:55.170538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1097/2202] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:46:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:46:55.189728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c418e12-7231-4932-89ba-b111a5703f7a admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:46:55.192442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-0c418e12-7231-4932-89ba-b111a5703f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:46:55.335133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:46:55.371058 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c418e12-7231-4932-89ba-b111a5703f7a admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:46:55.376492 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb4a76-68d7-467c-a6ce-2be0a2b4a4f4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c0fabecff2457e931c8c1c36681197', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:46:55.377171 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb4a76-68d7-467c-a6ce-2be0a2b4a4f4 service glance] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:46:55.379694 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb4a76-68d7-467c-a6ce-2be0a2b4a4f4 service glance] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:46:55.380021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb4a76-68d7-467c-a6ce-2be0a2b4a4f4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:46:55.395499 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb4a76-68d7-467c-a6ce-2be0a2b4a4f4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:46:55.398086 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb4a76-68d7-467c-a6ce-2be0a2b4a4f4 service glance] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:46:55.411255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1106/2203] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:46:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:47:05.166670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=15 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:47:05.406081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=6 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:47:21.099055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cdb1a-ae7e-4352-9c0b-3f73460afd84 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:21.100701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-237cdb1a-ae7e-4352-9c0b-3f73460afd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:21.179125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cdb1a-ae7e-4352-9c0b-3f73460afd84 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:21.185466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec33077-790b-43a9-a9a0-453b0603a78c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:21.186175 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec33077-790b-43a9-a9a0-453b0603a78c service placement] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:21.186489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec33077-790b-43a9-a9a0-453b0603a78c service placement] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:21.186788 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec33077-790b-43a9-a9a0-453b0603a78c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:21.188868 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5ec33077-790b-43a9-a9a0-453b0603a78c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:21.201824 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec33077-790b-43a9-a9a0-453b0603a78c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:21.203458 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec33077-790b-43a9-a9a0-453b0603a78c service placement] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:21.215323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1098/2204] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:47:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:47:21.571673 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f29a4f-ee24-4b5b-b4d1-88f8d6811bf4 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:21.572241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f29a4f-ee24-4b5b-b4d1-88f8d6811bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:21.572241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f29a4f-ee24-4b5b-b4d1-88f8d6811bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:21.593416 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-54f29a4f-ee24-4b5b-b4d1-88f8d6811bf4 None None] MFA Rules not processed for user `0216bf67389e4627bf953b7dcc97a88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:47:21.594686 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-54f29a4f-ee24-4b5b-b4d1-88f8d6811bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:21.657441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1107/2205] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 26 05:47:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:47:21.670113 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec33077-790b-43a9-a9a0-453b0603a78c service placement] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:21.673946 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5ec33077-790b-43a9-a9a0-453b0603a78c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:21.751236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec33077-790b-43a9-a9a0-453b0603a78c service placement] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:21.757156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3513c6-d9b4-45bf-84a7-98204fcddec5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f258aba84d4dd59ebe8de973ac65c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:21.757978 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3513c6-d9b4-45bf-84a7-98204fcddec5 service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:21.758396 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3513c6-d9b4-45bf-84a7-98204fcddec5 service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:21.758777 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3513c6-d9b4-45bf-84a7-98204fcddec5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:21.769052 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3513c6-d9b4-45bf-84a7-98204fcddec5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:21.770741 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3513c6-d9b4-45bf-84a7-98204fcddec5 service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:21.782310 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1099/2206] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 26 05:47:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:47:29.330817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f29a4f-ee24-4b5b-b4d1-88f8d6811bf4 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:29.433270 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f29a4f-ee24-4b5b-b4d1-88f8d6811bf4 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:29.439403 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228dc9c-6fc6-403b-8b40-1bf2bda01bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:29.440307 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228dc9c-6fc6-403b-8b40-1bf2bda01bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:29.440583 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228dc9c-6fc6-403b-8b40-1bf2bda01bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:29.440856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228dc9c-6fc6-403b-8b40-1bf2bda01bc0 admin admin] PATH_INFO: `/v3/users/54ef0b156bcf4e408eab746b7d9aa8ec` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:29.442094 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228dc9c-6fc6-403b-8b40-1bf2bda01bc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54ef0b156bcf4e408eab746b7d9aa8ec)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:29.444381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228dc9c-6fc6-403b-8b40-1bf2bda01bc0 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:29.565065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0228dc9c-6fc6-403b-8b40-1bf2bda01bc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54ef0b156bcf4e408eab746b7d9aa8ec'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:29.575727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0228dc9c-6fc6-403b-8b40-1bf2bda01bc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54ef0b156bcf4e408eab746b7d9aa8ec'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:29.576549 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0228dc9c-6fc6-403b-8b40-1bf2bda01bc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54ef0b156bcf4e408eab746b7d9aa8ec'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:29.577421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0228dc9c-6fc6-403b-8b40-1bf2bda01bc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54ef0b156bcf4e408eab746b7d9aa8ec'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:29.592783 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0228dc9c-6fc6-403b-8b40-1bf2bda01bc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54ef0b156bcf4e408eab746b7d9aa8ec'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:29.629424 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1108/2207] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:47:29 2022] DELETE /identity/v3/users/54ef0b156bcf4e408eab746b7d9aa8ec => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:47:29.632262 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80220da2-b8c5-49d5-89bc-795ec412947a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:29.632862 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80220da2-b8c5-49d5-89bc-795ec412947a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:29.633233 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80220da2-b8c5-49d5-89bc-795ec412947a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:29.676731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-80220da2-b8c5-49d5-89bc-795ec412947a None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:47:29.678091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-80220da2-b8c5-49d5-89bc-795ec412947a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:29.752654 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1100/2208] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:47:29.762664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228dc9c-6fc6-403b-8b40-1bf2bda01bc0 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:29.764150 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-0228dc9c-6fc6-403b-8b40-1bf2bda01bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:29.795117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:47:29.834934 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228dc9c-6fc6-403b-8b40-1bf2bda01bc0 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:29.841739 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00f7cc-c8b0-46b8-a25f-ae0961e86b7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:29.842371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00f7cc-c8b0-46b8-a25f-ae0961e86b7e service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:29.842676 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00f7cc-c8b0-46b8-a25f-ae0961e86b7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:29.842957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00f7cc-c8b0-46b8-a25f-ae0961e86b7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:29.854261 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00f7cc-c8b0-46b8-a25f-ae0961e86b7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:29.856103 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00f7cc-c8b0-46b8-a25f-ae0961e86b7e service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:29.878581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1109/2209] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:47:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:47:30.307182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80220da2-b8c5-49d5-89bc-795ec412947a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:30.309903 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-80220da2-b8c5-49d5-89bc-795ec412947a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:30.337522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80220da2-b8c5-49d5-89bc-795ec412947a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:30.350416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b950f-867e-4150-aac6-5d38d37b3200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:30.351061 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b950f-867e-4150-aac6-5d38d37b3200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:30.351374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b950f-867e-4150-aac6-5d38d37b3200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:30.351649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b950f-867e-4150-aac6-5d38d37b3200 admin admin] PATH_INFO: `/v3/projects/895bbbd2094d4d47b52753223a6db939` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:30.353004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b950f-867e-4150-aac6-5d38d37b3200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=895bbbd2094d4d47b52753223a6db939)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:30.355950 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b950f-867e-4150-aac6-5d38d37b3200 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:30.377712 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:47:30.422576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ff9b950f-867e-4150-aac6-5d38d37b3200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '895bbbd2094d4d47b52753223a6db939', 'request_id': 'req-ff9b950f-867e-4150-aac6-5d38d37b3200'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:30.432347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ff9b950f-867e-4150-aac6-5d38d37b3200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '895bbbd2094d4d47b52753223a6db939', 'request_id': 'req-ff9b950f-867e-4150-aac6-5d38d37b3200'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:30.453182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ff9b950f-867e-4150-aac6-5d38d37b3200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '895bbbd2094d4d47b52753223a6db939', 'request_id': 'req-ff9b950f-867e-4150-aac6-5d38d37b3200'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:30.472683 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ff9b950f-867e-4150-aac6-5d38d37b3200 admin admin] The token cache is being invalidate because project 895bbbd2094d4d47b52753223a6db939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:47:30.472683 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ff9b950f-867e-4150-aac6-5d38d37b3200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:30.473542 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ff9b950f-867e-4150-aac6-5d38d37b3200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:30.475907 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1101/2210] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:47:30 2022] DELETE /identity/v3/projects/895bbbd2094d4d47b52753223a6db939 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:47:30.483600 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3743bba-b38f-4268-a506-203250cc23c5 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:30.484216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3743bba-b38f-4268-a506-203250cc23c5 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:30.484216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3743bba-b38f-4268-a506-203250cc23c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:30.510963 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d3743bba-b38f-4268-a506-203250cc23c5 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:47:30.512838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d3743bba-b38f-4268-a506-203250cc23c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:30.594171 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1110/2211] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:47:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:47:30.605301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cadb5-f3ec-40b2-9bbd-b3e7562d56fa None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:30.605567 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cadb5-f3ec-40b2-9bbd-b3e7562d56fa None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:30.605894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cadb5-f3ec-40b2-9bbd-b3e7562d56fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:30.639405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7b9cadb5-f3ec-40b2-9bbd-b3e7562d56fa None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:47:30.640899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7b9cadb5-f3ec-40b2-9bbd-b3e7562d56fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:30.650866 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1102/2212] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:47:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:47:30.655963 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3743bba-b38f-4268-a506-203250cc23c5 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:30.687297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3743bba-b38f-4268-a506-203250cc23c5 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:30.705071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c2477-e5a7-496b-8fe1-bba55d2b145f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:30.706252 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c2477-e5a7-496b-8fe1-bba55d2b145f admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:30.706636 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c2477-e5a7-496b-8fe1-bba55d2b145f admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:30.706974 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c2477-e5a7-496b-8fe1-bba55d2b145f admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:30.707670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c2477-e5a7-496b-8fe1-bba55d2b145f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:47:30.707978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c2477-e5a7-496b-8fe1-bba55d2b145f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:30.709723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c2477-e5a7-496b-8fe1-bba55d2b145f admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:30.733462 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1111/2213] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:47:30.733940 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cadb5-f3ec-40b2-9bbd-b3e7562d56fa None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:30.746377 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cadb5-f3ec-40b2-9bbd-b3e7562d56fa None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:30.756714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24b0a5-a92c-4778-a21b-4034ae69495c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:30.756714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24b0a5-a92c-4778-a21b-4034ae69495c admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:30.756714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24b0a5-a92c-4778-a21b-4034ae69495c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:30.756714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24b0a5-a92c-4778-a21b-4034ae69495c admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:30.757244 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24b0a5-a92c-4778-a21b-4034ae69495c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:30.757910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24b0a5-a92c-4778-a21b-4034ae69495c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:30.813185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1103/2214] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:47:30 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:47:30.815781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c2477-e5a7-496b-8fe1-bba55d2b145f admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:30.834329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c2477-e5a7-496b-8fe1-bba55d2b145f admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:30.936907 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68820970-b302-4210-9133-375e208d2c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:30.937873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68820970-b302-4210-9133-375e208d2c80 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:30.938248 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68820970-b302-4210-9133-375e208d2c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:30.938597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68820970-b302-4210-9133-375e208d2c80 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:30.939392 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68820970-b302-4210-9133-375e208d2c80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:30.941569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68820970-b302-4210-9133-375e208d2c80 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:30.990066 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1112/2215] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:47:30 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:47:30.991616 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24b0a5-a92c-4778-a21b-4034ae69495c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:31.020047 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24b0a5-a92c-4778-a21b-4034ae69495c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:31.035859 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd201c-7242-4114-851f-1deb43fe6286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:31.036606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd201c-7242-4114-851f-1deb43fe6286 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:31.036980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd201c-7242-4114-851f-1deb43fe6286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:31.037390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd201c-7242-4114-851f-1deb43fe6286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:31.038004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd201c-7242-4114-851f-1deb43fe6286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:31.039675 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd201c-7242-4114-851f-1deb43fe6286 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:31.051753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1104/2216] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:47:31.055867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68820970-b302-4210-9133-375e208d2c80 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:31.066712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68820970-b302-4210-9133-375e208d2c80 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:31.072731 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ebf2b-759a-4baa-8117-815c04f878c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:31.073469 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ebf2b-759a-4baa-8117-815c04f878c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:31.073750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ebf2b-759a-4baa-8117-815c04f878c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:31.074022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ebf2b-759a-4baa-8117-815c04f878c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:31.074951 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ebf2b-759a-4baa-8117-815c04f878c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:31.076745 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ebf2b-759a-4baa-8117-815c04f878c0 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:31.094552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1113/2217] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:47:31.099038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd201c-7242-4114-851f-1deb43fe6286 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:31.114738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd201c-7242-4114-851f-1deb43fe6286 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:31.120170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d3008-d70f-44ee-9c01-2c15aca7c69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:31.120901 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d3008-d70f-44ee-9c01-2c15aca7c69f admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:31.121227 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d3008-d70f-44ee-9c01-2c15aca7c69f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:31.121540 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d3008-d70f-44ee-9c01-2c15aca7c69f admin admin] PATH_INFO: `/v3/projects/fc37f8f28d2543528dd268254c91a8c0/users/9d5de49661aa4ab3bdef6485b6422cfa/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:31.143783 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d3008-d70f-44ee-9c01-2c15aca7c69f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc37f8f28d2543528dd268254c91a8c0, user_id=9d5de49661aa4ab3bdef6485b6422cfa, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:31.146555 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d3008-d70f-44ee-9c01-2c15aca7c69f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:31.166437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1105/2218] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:47:31 2022] PUT /identity/v3/projects/fc37f8f28d2543528dd268254c91a8c0/users/9d5de49661aa4ab3bdef6485b6422cfa/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:47:31.177702 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e280b22-2f9a-41cc-89a0-68b160ca62e8 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:31.178182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e280b22-2f9a-41cc-89a0-68b160ca62e8 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:31.178552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e280b22-2f9a-41cc-89a0-68b160ca62e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:31.212779 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-0e280b22-2f9a-41cc-89a0-68b160ca62e8 None None] MFA Rules not processed for user `9d5de49661aa4ab3bdef6485b6422cfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:47:31.214292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-0e280b22-2f9a-41cc-89a0-68b160ca62e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:31.242020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:47:31.304157 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1114/2219] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:47:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:47:31.326139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d3008-d70f-44ee-9c01-2c15aca7c69f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:31.329062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-521d3008-d70f-44ee-9c01-2c15aca7c69f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:31.407101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d3008-d70f-44ee-9c01-2c15aca7c69f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:31.411329 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c9d42-081e-4663-9517-f409d4109d2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:31.412093 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c9d42-081e-4663-9517-f409d4109d2f service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:31.412443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c9d42-081e-4663-9517-f409d4109d2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:31.412723 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c9d42-081e-4663-9517-f409d4109d2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:31.424061 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c9d42-081e-4663-9517-f409d4109d2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:31.425755 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c9d42-081e-4663-9517-f409d4109d2f service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:31.437715 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1106/2220] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:47:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:47:41.227999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7be4bb-0139-4599-a201-615964c0aaa9 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:41.228624 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7be4bb-0139-4599-a201-615964c0aaa9 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:41.228624 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7be4bb-0139-4599-a201-615964c0aaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:41.248787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-0a7be4bb-0139-4599-a201-615964c0aaa9 None None] MFA Rules not processed for user `0216bf67389e4627bf953b7dcc97a88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:47:41.250039 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-0a7be4bb-0139-4599-a201-615964c0aaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:41.314338 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1115/2221] 10.222.0.57 () {58 vars in 967 bytes} [Mon Sep 26 05:47:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:47:41.328447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c9d42-081e-4663-9517-f409d4109d2f service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:41.330128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-672c9d42-081e-4663-9517-f409d4109d2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:41.386554 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:47:41.422531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c9d42-081e-4663-9517-f409d4109d2f service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:41.424008 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55354ba9-ca92-49da-8fe2-b31125fda0a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f258aba84d4dd59ebe8de973ac65c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:41.424647 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55354ba9-ca92-49da-8fe2-b31125fda0a2 service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:41.425088 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55354ba9-ca92-49da-8fe2-b31125fda0a2 service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:41.425464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55354ba9-ca92-49da-8fe2-b31125fda0a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:41.435949 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55354ba9-ca92-49da-8fe2-b31125fda0a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:41.437706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55354ba9-ca92-49da-8fe2-b31125fda0a2 service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:41.447638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1107/2222] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 26 05:47:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:47:48.823573 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7be4bb-0139-4599-a201-615964c0aaa9 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:48.825279 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-0a7be4bb-0139-4599-a201-615964c0aaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:48.898676 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7be4bb-0139-4599-a201-615964c0aaa9 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:48.903990 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b65c4b-6c56-434b-8176-6d64d97b3b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:48.904641 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b65c4b-6c56-434b-8176-6d64d97b3b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:48.904915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b65c4b-6c56-434b-8176-6d64d97b3b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:48.905213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b65c4b-6c56-434b-8176-6d64d97b3b5a admin admin] PATH_INFO: `/v3/users/04c7152003c4491abea8305c26f366ed` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:48.906362 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b65c4b-6c56-434b-8176-6d64d97b3b5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=04c7152003c4491abea8305c26f366ed)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:48.908274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b65c4b-6c56-434b-8176-6d64d97b3b5a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:48.959110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-35b65c4b-6c56-434b-8176-6d64d97b3b5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04c7152003c4491abea8305c26f366ed'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:48.968078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-35b65c4b-6c56-434b-8176-6d64d97b3b5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04c7152003c4491abea8305c26f366ed'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:48.968788 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-35b65c4b-6c56-434b-8176-6d64d97b3b5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04c7152003c4491abea8305c26f366ed'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:48.969441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-35b65c4b-6c56-434b-8176-6d64d97b3b5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04c7152003c4491abea8305c26f366ed'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:48.979229 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-35b65c4b-6c56-434b-8176-6d64d97b3b5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04c7152003c4491abea8305c26f366ed'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:48.984661 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:47:49.000644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1116/2223] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:47:48 2022] DELETE /identity/v3/users/04c7152003c4491abea8305c26f366ed => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:47:49.194910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55354ba9-ca92-49da-8fe2-b31125fda0a2 service swift] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:49.211650 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55354ba9-ca92-49da-8fe2-b31125fda0a2 service swift] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:49.277020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90442f2-1fd5-4264-bc95-d2f65bc4ea46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:49.277703 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90442f2-1fd5-4264-bc95-d2f65bc4ea46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:49.277972 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90442f2-1fd5-4264-bc95-d2f65bc4ea46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:49.278239 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90442f2-1fd5-4264-bc95-d2f65bc4ea46 admin admin] PATH_INFO: `/v3/projects/ee638507f9b0415f94f7c54e0e3db8cd` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:49.279445 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90442f2-1fd5-4264-bc95-d2f65bc4ea46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee638507f9b0415f94f7c54e0e3db8cd)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:49.281391 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90442f2-1fd5-4264-bc95-d2f65bc4ea46 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:49.344935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c90442f2-1fd5-4264-bc95-d2f65bc4ea46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee638507f9b0415f94f7c54e0e3db8cd', 'request_id': 'req-c90442f2-1fd5-4264-bc95-d2f65bc4ea46'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:49.352566 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c90442f2-1fd5-4264-bc95-d2f65bc4ea46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee638507f9b0415f94f7c54e0e3db8cd', 'request_id': 'req-c90442f2-1fd5-4264-bc95-d2f65bc4ea46'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:49.361202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c90442f2-1fd5-4264-bc95-d2f65bc4ea46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee638507f9b0415f94f7c54e0e3db8cd', 'request_id': 'req-c90442f2-1fd5-4264-bc95-d2f65bc4ea46'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:49.371729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c90442f2-1fd5-4264-bc95-d2f65bc4ea46 admin admin] The token cache is being invalidate because project ee638507f9b0415f94f7c54e0e3db8cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:47:49.372015 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c90442f2-1fd5-4264-bc95-d2f65bc4ea46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:49.372656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-c90442f2-1fd5-4264-bc95-d2f65bc4ea46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:47:49.375975 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1108/2224] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:47:49 2022] DELETE /identity/v3/projects/ee638507f9b0415f94f7c54e0e3db8cd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:47:49.381305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f3f70-583a-4ddb-b2c4-9231da72b3ab None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:49.381599 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f3f70-583a-4ddb-b2c4-9231da72b3ab None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:49.381881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f3f70-583a-4ddb-b2c4-9231da72b3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:49.404092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-de9f3f70-583a-4ddb-b2c4-9231da72b3ab None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:47:49.405392 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-de9f3f70-583a-4ddb-b2c4-9231da72b3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:49.476933 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1117/2225] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:47:49.484988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d16061-dbbd-4dcf-84a7-dbde6a0ac903 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:49.485314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d16061-dbbd-4dcf-84a7-dbde6a0ac903 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:49.485609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d16061-dbbd-4dcf-84a7-dbde6a0ac903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:49.506600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d2d16061-dbbd-4dcf-84a7-dbde6a0ac903 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:47:49.507800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d2d16061-dbbd-4dcf-84a7-dbde6a0ac903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:49.517000 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1109/2226] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:47:49.523491 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f3f70-583a-4ddb-b2c4-9231da72b3ab None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:49.535848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f3f70-583a-4ddb-b2c4-9231da72b3ab None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:49.543687 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143bd668-47d4-4c29-a689-edd94f70c05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:49.544687 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bd668-47d4-4c29-a689-edd94f70c05b admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:49.545118 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bd668-47d4-4c29-a689-edd94f70c05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:49.545491 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bd668-47d4-4c29-a689-edd94f70c05b admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:49.546309 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143bd668-47d4-4c29-a689-edd94f70c05b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:47:49.546641 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143bd668-47d4-4c29-a689-edd94f70c05b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:49.548847 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143bd668-47d4-4c29-a689-edd94f70c05b admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:49.566348 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1118/2227] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:47:49.571570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d16061-dbbd-4dcf-84a7-dbde6a0ac903 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:49.582273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d16061-dbbd-4dcf-84a7-dbde6a0ac903 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:49.588781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718aa61-2f20-4afb-b4f9-caaa73eafa5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:49.589416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718aa61-2f20-4afb-b4f9-caaa73eafa5a admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:49.589722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718aa61-2f20-4afb-b4f9-caaa73eafa5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:49.589990 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718aa61-2f20-4afb-b4f9-caaa73eafa5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:49.590626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718aa61-2f20-4afb-b4f9-caaa73eafa5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:49.592524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718aa61-2f20-4afb-b4f9-caaa73eafa5a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:49.630902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1110/2228] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:47:49 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:47:49.635148 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143bd668-47d4-4c29-a689-edd94f70c05b admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:49.645572 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143bd668-47d4-4c29-a689-edd94f70c05b admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:49.703987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3047f-852a-4681-8e2f-64ddda6f9d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:49.704581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3047f-852a-4681-8e2f-64ddda6f9d3e admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:49.704864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3047f-852a-4681-8e2f-64ddda6f9d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:49.705156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3047f-852a-4681-8e2f-64ddda6f9d3e admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:49.705840 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3047f-852a-4681-8e2f-64ddda6f9d3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:49.707806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3047f-852a-4681-8e2f-64ddda6f9d3e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:49.731812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1119/2229] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:47:49 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:47:49.736436 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718aa61-2f20-4afb-b4f9-caaa73eafa5a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:49.749390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718aa61-2f20-4afb-b4f9-caaa73eafa5a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:49.755867 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a307cc-4943-4b86-90d6-3b5423e22b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:49.756799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a307cc-4943-4b86-90d6-3b5423e22b65 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:49.757216 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a307cc-4943-4b86-90d6-3b5423e22b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:49.757586 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a307cc-4943-4b86-90d6-3b5423e22b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:49.758610 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a307cc-4943-4b86-90d6-3b5423e22b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:49.760721 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a307cc-4943-4b86-90d6-3b5423e22b65 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:49.772408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1111/2230] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:47:49.777355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3047f-852a-4681-8e2f-64ddda6f9d3e admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:49.790279 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3047f-852a-4681-8e2f-64ddda6f9d3e admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:49.795080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3402a5-f516-4d87-8c2c-4d91fa298570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:49.795670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3402a5-f516-4d87-8c2c-4d91fa298570 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:49.795943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3402a5-f516-4d87-8c2c-4d91fa298570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:49.796210 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3402a5-f516-4d87-8c2c-4d91fa298570 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:49.796844 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3402a5-f516-4d87-8c2c-4d91fa298570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:49.798463 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3402a5-f516-4d87-8c2c-4d91fa298570 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:49.810365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1120/2231] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:47:49.814692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a307cc-4943-4b86-90d6-3b5423e22b65 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:49.825730 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a307cc-4943-4b86-90d6-3b5423e22b65 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:49.831036 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba6abf-f700-4a5c-933d-85e63257092b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:49.831709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba6abf-f700-4a5c-933d-85e63257092b admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:49.831970 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba6abf-f700-4a5c-933d-85e63257092b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:49.832260 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba6abf-f700-4a5c-933d-85e63257092b admin admin] PATH_INFO: `/v3/projects/0b095f1ee00e46cfabebc2dc34992997/users/3cab2568112f46bca576f17916887aa7/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:49.849814 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba6abf-f700-4a5c-933d-85e63257092b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b095f1ee00e46cfabebc2dc34992997, user_id=3cab2568112f46bca576f17916887aa7, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:49.852496 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba6abf-f700-4a5c-933d-85e63257092b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:49.862721 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:47:49.869085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1112/2232] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:47:49 2022] PUT /identity/v3/projects/0b095f1ee00e46cfabebc2dc34992997/users/3cab2568112f46bca576f17916887aa7/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:47:49.874554 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18eee5-5742-4bfe-a8a9-ec170d5dd2ef None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:49.875056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18eee5-5742-4bfe-a8a9-ec170d5dd2ef None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:49.875153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18eee5-5742-4bfe-a8a9-ec170d5dd2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:49.894439 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-3c18eee5-5742-4bfe-a8a9-ec170d5dd2ef None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:47:49.895649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-3c18eee5-5742-4bfe-a8a9-ec170d5dd2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:49.951778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:47:49.966457 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1121/2233] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:47:49.976410 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba6abf-f700-4a5c-933d-85e63257092b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:49.978181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-20ba6abf-f700-4a5c-933d-85e63257092b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:50.046984 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba6abf-f700-4a5c-933d-85e63257092b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:50.051324 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefdb86-e4d6-4dcb-804c-30de58bbfa3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:50.051945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefdb86-e4d6-4dcb-804c-30de58bbfa3a service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:50.052214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefdb86-e4d6-4dcb-804c-30de58bbfa3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:50.052482 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefdb86-e4d6-4dcb-804c-30de58bbfa3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:50.063643 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfefdb86-e4d6-4dcb-804c-30de58bbfa3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:50.065157 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfefdb86-e4d6-4dcb-804c-30de58bbfa3a service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:50.076521 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1113/2234] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:47:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:47:54.753393 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed123ccb-02f9-46da-b708-7106072471de None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:54.753913 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed123ccb-02f9-46da-b708-7106072471de None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:54.754004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed123ccb-02f9-46da-b708-7106072471de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:54.788251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-ed123ccb-02f9-46da-b708-7106072471de None None] MFA Rules not processed for user `3cab2568112f46bca576f17916887aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:47:54.789567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ed123ccb-02f9-46da-b708-7106072471de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:47:54.859377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1122/2235] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:47:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:47:54.866234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefdb86-e4d6-4dcb-804c-30de58bbfa3a service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:54.877830 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefdb86-e4d6-4dcb-804c-30de58bbfa3a service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:54.883923 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea243a37-2011-48ce-a759-989763e24680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:54.884632 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea243a37-2011-48ce-a759-989763e24680 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:54.884982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea243a37-2011-48ce-a759-989763e24680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:54.885361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea243a37-2011-48ce-a759-989763e24680 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:54.886027 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea243a37-2011-48ce-a759-989763e24680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:54.890367 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea243a37-2011-48ce-a759-989763e24680 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:54.930764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1114/2236] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:47:54 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:47:54.935133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed123ccb-02f9-46da-b708-7106072471de None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:54.947133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed123ccb-02f9-46da-b708-7106072471de None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:55.034479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929297f-252f-4f6f-8a99-f5ff82da1ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:55.035375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929297f-252f-4f6f-8a99-f5ff82da1ebe admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:55.035789 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929297f-252f-4f6f-8a99-f5ff82da1ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:55.036202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929297f-252f-4f6f-8a99-f5ff82da1ebe admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:55.037345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929297f-252f-4f6f-8a99-f5ff82da1ebe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:55.041191 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929297f-252f-4f6f-8a99-f5ff82da1ebe admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:55.069437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1123/2237] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:47:54 2022] POST /identity/v3/users => generated 298 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:47:55.073727 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea243a37-2011-48ce-a759-989763e24680 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:55.089792 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea243a37-2011-48ce-a759-989763e24680 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:55.094604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acab91-8fd0-490b-a089-4bd4e680db4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:55.095225 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acab91-8fd0-490b-a089-4bd4e680db4a admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:55.095500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acab91-8fd0-490b-a089-4bd4e680db4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:55.095771 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acab91-8fd0-490b-a089-4bd4e680db4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:55.096413 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acab91-8fd0-490b-a089-4bd4e680db4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:55.098070 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acab91-8fd0-490b-a089-4bd4e680db4a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:55.110871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1115/2238] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:47:55.115432 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929297f-252f-4f6f-8a99-f5ff82da1ebe admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:55.125761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929297f-252f-4f6f-8a99-f5ff82da1ebe admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:55.130639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d436924-8db0-4f6a-87ba-335e3100a2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:55.131320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d436924-8db0-4f6a-87ba-335e3100a2bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:55.131617 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d436924-8db0-4f6a-87ba-335e3100a2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:55.131890 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d436924-8db0-4f6a-87ba-335e3100a2bb admin admin] PATH_INFO: `/v3/projects/d7219cb9801c4589b04ed212e3e06e08/users/9b88e498fb9049af99d3c802af19bb73/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:55.148992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d436924-8db0-4f6a-87ba-335e3100a2bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7219cb9801c4589b04ed212e3e06e08, user_id=9b88e498fb9049af99d3c802af19bb73, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:55.153233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d436924-8db0-4f6a-87ba-335e3100a2bb admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:55.168225 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1124/2239] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:47:55 2022] PUT /identity/v3/projects/d7219cb9801c4589b04ed212e3e06e08/users/9b88e498fb9049af99d3c802af19bb73/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:47:55.172813 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acab91-8fd0-490b-a089-4bd4e680db4a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:55.185904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acab91-8fd0-490b-a089-4bd4e680db4a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:55.280727 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a43283-457e-495f-bbba-175077a07119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:55.281774 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a43283-457e-495f-bbba-175077a07119 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:55.282000 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a43283-457e-495f-bbba-175077a07119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:55.282266 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a43283-457e-495f-bbba-175077a07119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:55.282933 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a43283-457e-495f-bbba-175077a07119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:55.284560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a43283-457e-495f-bbba-175077a07119 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:55.295743 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1116/2240] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:47:55.300732 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d436924-8db0-4f6a-87ba-335e3100a2bb admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:47:55.313046 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d436924-8db0-4f6a-87ba-335e3100a2bb admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:47:55.323359 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276f22b-7188-4d65-8443-361c671bbb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:47:55.324204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276f22b-7188-4d65-8443-361c671bbb04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:47:55.324505 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276f22b-7188-4d65-8443-361c671bbb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:47:55.324867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276f22b-7188-4d65-8443-361c671bbb04 admin admin] PATH_INFO: `/v3/projects/d7219cb9801c4589b04ed212e3e06e08/users/9b88e498fb9049af99d3c802af19bb73/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:47:55.327658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276f22b-7188-4d65-8443-361c671bbb04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7219cb9801c4589b04ed212e3e06e08, user_id=9b88e498fb9049af99d3c802af19bb73, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:47:55.330680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276f22b-7188-4d65-8443-361c671bbb04 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:47:55.343563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1125/2241] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:47:55 2022] PUT /identity/v3/projects/d7219cb9801c4589b04ed212e3e06e08/users/9b88e498fb9049af99d3c802af19bb73/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:48:00.294743 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4d48a-3cc4-4a3d-8d92-37988f592e83 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:00.295665 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4d48a-3cc4-4a3d-8d92-37988f592e83 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:00.295665 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4d48a-3cc4-4a3d-8d92-37988f592e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:00.329503 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c1d4d48a-3cc4-4a3d-8d92-37988f592e83 None None] MFA Rules not processed for user `9b88e498fb9049af99d3c802af19bb73`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:48:00.330754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c1d4d48a-3cc4-4a3d-8d92-37988f592e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:00.390725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:48:00.414623 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1117/2242] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:48:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:48:00.439599 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276f22b-7188-4d65-8443-361c671bbb04 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:00.441610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c276f22b-7188-4d65-8443-361c671bbb04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:00.497116 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:48:00.522147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276f22b-7188-4d65-8443-361c671bbb04 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:00.532196 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ec4ce-dfb3-45e9-83f3-9b9a2049e874 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c0fabecff2457e931c8c1c36681197', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:00.533555 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ec4ce-dfb3-45e9-83f3-9b9a2049e874 service glance] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:00.534012 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ec4ce-dfb3-45e9-83f3-9b9a2049e874 service glance] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:00.534625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ec4ce-dfb3-45e9-83f3-9b9a2049e874 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:00.617531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ec4ce-dfb3-45e9-83f3-9b9a2049e874 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:00.619421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ec4ce-dfb3-45e9-83f3-9b9a2049e874 service glance] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:00.632068 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1126/2243] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:48:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:48:00.883073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab904976-864a-4aad-a1d1-2e92f7b7b153 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:00.883656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab904976-864a-4aad-a1d1-2e92f7b7b153 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:00.884150 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab904976-864a-4aad-a1d1-2e92f7b7b153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:00.911381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ab904976-864a-4aad-a1d1-2e92f7b7b153 None None] MFA Rules not processed for user `0216bf67389e4627bf953b7dcc97a88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:48:00.912679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ab904976-864a-4aad-a1d1-2e92f7b7b153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:00.992024 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1118/2244] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 26 05:48:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:48:01.016801 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ec4ce-dfb3-45e9-83f3-9b9a2049e874 service glance] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:01.018652 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-383ec4ce-dfb3-45e9-83f3-9b9a2049e874 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:01.120028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ec4ce-dfb3-45e9-83f3-9b9a2049e874 service glance] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:01.127471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be94fa1-a1d9-4b0b-913c-cdcc80e9972e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f258aba84d4dd59ebe8de973ac65c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:01.128488 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be94fa1-a1d9-4b0b-913c-cdcc80e9972e service swift] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:01.129063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be94fa1-a1d9-4b0b-913c-cdcc80e9972e service swift] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:01.129405 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be94fa1-a1d9-4b0b-913c-cdcc80e9972e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:01.139507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be94fa1-a1d9-4b0b-913c-cdcc80e9972e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:01.141677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be94fa1-a1d9-4b0b-913c-cdcc80e9972e service swift] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:01.151150 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1127/2245] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 26 05:48:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:48:02.635977 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab904976-864a-4aad-a1d1-2e92f7b7b153 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:02.637664 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ab904976-864a-4aad-a1d1-2e92f7b7b153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:02.756260 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab904976-864a-4aad-a1d1-2e92f7b7b153 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:02.776336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03851924-ac32-4b3f-9db9-95917c818a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:02.777742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03851924-ac32-4b3f-9db9-95917c818a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:02.778428 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03851924-ac32-4b3f-9db9-95917c818a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:02.780847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03851924-ac32-4b3f-9db9-95917c818a91 admin admin] PATH_INFO: `/v3/users/9d5de49661aa4ab3bdef6485b6422cfa` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:02.782835 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03851924-ac32-4b3f-9db9-95917c818a91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d5de49661aa4ab3bdef6485b6422cfa)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:02.786942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03851924-ac32-4b3f-9db9-95917c818a91 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:02.903237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-03851924-ac32-4b3f-9db9-95917c818a91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d5de49661aa4ab3bdef6485b6422cfa'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:02.912721 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-03851924-ac32-4b3f-9db9-95917c818a91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d5de49661aa4ab3bdef6485b6422cfa'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:02.913737 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-03851924-ac32-4b3f-9db9-95917c818a91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d5de49661aa4ab3bdef6485b6422cfa'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:02.915390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-03851924-ac32-4b3f-9db9-95917c818a91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d5de49661aa4ab3bdef6485b6422cfa'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:02.955909 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-03851924-ac32-4b3f-9db9-95917c818a91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d5de49661aa4ab3bdef6485b6422cfa'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:02.979138 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1119/2246] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:48:02 2022] DELETE /identity/v3/users/9d5de49661aa4ab3bdef6485b6422cfa => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:48:02.983875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50ff97-d98f-46ad-9ab3-cdef308686cf None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:02.984419 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50ff97-d98f-46ad-9ab3-cdef308686cf None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:02.984419 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50ff97-d98f-46ad-9ab3-cdef308686cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:03.007429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-cb50ff97-d98f-46ad-9ab3-cdef308686cf None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:48:03.008677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-cb50ff97-d98f-46ad-9ab3-cdef308686cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:03.092354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1128/2247] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:48:03.102248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03851924-ac32-4b3f-9db9-95917c818a91 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:03.103897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-03851924-ac32-4b3f-9db9-95917c818a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:03.123489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:48:03.192507 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03851924-ac32-4b3f-9db9-95917c818a91 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:03.197093 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2baf4fc-6433-4abd-9932-ab07ac527266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:03.197781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baf4fc-6433-4abd-9932-ab07ac527266 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:03.198062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baf4fc-6433-4abd-9932-ab07ac527266 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:03.198326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baf4fc-6433-4abd-9932-ab07ac527266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:03.208911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2baf4fc-6433-4abd-9932-ab07ac527266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:03.210392 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2baf4fc-6433-4abd-9932-ab07ac527266 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:03.238199 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1120/2248] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:48:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:48:03.624631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50ff97-d98f-46ad-9ab3-cdef308686cf None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:03.628283 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-cb50ff97-d98f-46ad-9ab3-cdef308686cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:03.647630 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50ff97-d98f-46ad-9ab3-cdef308686cf None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:03.670047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf3801-d2e5-448b-82c7-76c2587086d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:03.672020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf3801-d2e5-448b-82c7-76c2587086d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:03.672479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf3801-d2e5-448b-82c7-76c2587086d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:03.672945 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf3801-d2e5-448b-82c7-76c2587086d2 admin admin] PATH_INFO: `/v3/projects/fc37f8f28d2543528dd268254c91a8c0` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:03.678841 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf3801-d2e5-448b-82c7-76c2587086d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc37f8f28d2543528dd268254c91a8c0)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:03.681091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf3801-d2e5-448b-82c7-76c2587086d2 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:03.711542 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:48:03.762202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d7cf3801-d2e5-448b-82c7-76c2587086d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc37f8f28d2543528dd268254c91a8c0', 'request_id': 'req-d7cf3801-d2e5-448b-82c7-76c2587086d2'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:03.769410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d7cf3801-d2e5-448b-82c7-76c2587086d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc37f8f28d2543528dd268254c91a8c0', 'request_id': 'req-d7cf3801-d2e5-448b-82c7-76c2587086d2'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:03.778810 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d7cf3801-d2e5-448b-82c7-76c2587086d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc37f8f28d2543528dd268254c91a8c0', 'request_id': 'req-d7cf3801-d2e5-448b-82c7-76c2587086d2'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:03.794576 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d7cf3801-d2e5-448b-82c7-76c2587086d2 admin admin] The token cache is being invalidate because project fc37f8f28d2543528dd268254c91a8c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:48:03.794870 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d7cf3801-d2e5-448b-82c7-76c2587086d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:03.795648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d7cf3801-d2e5-448b-82c7-76c2587086d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:03.798518 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1129/2249] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:48:03 2022] DELETE /identity/v3/projects/fc37f8f28d2543528dd268254c91a8c0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:48:03.803736 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f088fb-3178-4f5c-945c-23dd2fc1b789 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:03.804231 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f088fb-3178-4f5c-945c-23dd2fc1b789 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:03.804324 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f088fb-3178-4f5c-945c-23dd2fc1b789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:03.828850 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d0f088fb-3178-4f5c-945c-23dd2fc1b789 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:48:03.830171 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d0f088fb-3178-4f5c-945c-23dd2fc1b789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:03.905414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1121/2250] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:48:03.910716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824eced-46d4-48d8-959a-145ac8ca4c5b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:03.911269 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824eced-46d4-48d8-959a-145ac8ca4c5b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:03.911631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824eced-46d4-48d8-959a-145ac8ca4c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:03.939220 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-0824eced-46d4-48d8-959a-145ac8ca4c5b None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:48:03.940793 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-0824eced-46d4-48d8-959a-145ac8ca4c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:03.954080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1130/2251] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:48:03.955479 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f088fb-3178-4f5c-945c-23dd2fc1b789 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:03.973958 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f088fb-3178-4f5c-945c-23dd2fc1b789 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:03.979674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8570214-b43a-45ed-94a2-a635c9ba335e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:03.980468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8570214-b43a-45ed-94a2-a635c9ba335e admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:03.980808 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8570214-b43a-45ed-94a2-a635c9ba335e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:03.981168 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8570214-b43a-45ed-94a2-a635c9ba335e admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:03.981925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8570214-b43a-45ed-94a2-a635c9ba335e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:48:03.982292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8570214-b43a-45ed-94a2-a635c9ba335e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:03.984369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8570214-b43a-45ed-94a2-a635c9ba335e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:04.001218 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1122/2252] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:48:04.002440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824eced-46d4-48d8-959a-145ac8ca4c5b None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:04.016594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824eced-46d4-48d8-959a-145ac8ca4c5b None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:04.022905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201e414-e892-4bb1-895e-97ca2d028ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:04.023607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201e414-e892-4bb1-895e-97ca2d028ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:04.023964 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201e414-e892-4bb1-895e-97ca2d028ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:04.024304 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201e414-e892-4bb1-895e-97ca2d028ac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:04.027806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201e414-e892-4bb1-895e-97ca2d028ac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:04.029965 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201e414-e892-4bb1-895e-97ca2d028ac5 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:04.086715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1131/2253] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:48:04 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:48:04.087975 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8570214-b43a-45ed-94a2-a635c9ba335e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:04.100068 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8570214-b43a-45ed-94a2-a635c9ba335e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:04.179196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f91fc0-3217-43c9-84aa-67132dbfff48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:04.179833 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f91fc0-3217-43c9-84aa-67132dbfff48 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:04.180110 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f91fc0-3217-43c9-84aa-67132dbfff48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:04.180380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f91fc0-3217-43c9-84aa-67132dbfff48 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:04.181043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f91fc0-3217-43c9-84aa-67132dbfff48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:04.183084 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f91fc0-3217-43c9-84aa-67132dbfff48 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:04.210543 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1123/2254] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:48:04 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:48:04.215493 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201e414-e892-4bb1-895e-97ca2d028ac5 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:04.235491 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201e414-e892-4bb1-895e-97ca2d028ac5 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:04.240271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d522eb-b0b5-4610-9260-194d1361ac83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:04.240991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d522eb-b0b5-4610-9260-194d1361ac83 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:04.241374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d522eb-b0b5-4610-9260-194d1361ac83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:04.241738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d522eb-b0b5-4610-9260-194d1361ac83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:04.242481 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d522eb-b0b5-4610-9260-194d1361ac83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:04.244513 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d522eb-b0b5-4610-9260-194d1361ac83 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:04.259728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1132/2255] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:48:04.264107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f91fc0-3217-43c9-84aa-67132dbfff48 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:04.289178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f91fc0-3217-43c9-84aa-67132dbfff48 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:04.292435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b6d16-1f94-4138-8495-b735e750f10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:04.293627 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b6d16-1f94-4138-8495-b735e750f10b admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:04.294173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b6d16-1f94-4138-8495-b735e750f10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:04.294697 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b6d16-1f94-4138-8495-b735e750f10b admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:04.295843 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b6d16-1f94-4138-8495-b735e750f10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:04.299611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b6d16-1f94-4138-8495-b735e750f10b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:04.318447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1124/2256] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:48:04.325219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d522eb-b0b5-4610-9260-194d1361ac83 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:04.336552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d522eb-b0b5-4610-9260-194d1361ac83 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:04.342158 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d9683-9825-44fe-9a5f-74bc9a334486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:04.342916 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d9683-9825-44fe-9a5f-74bc9a334486 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:04.343738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d9683-9825-44fe-9a5f-74bc9a334486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:04.343738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d9683-9825-44fe-9a5f-74bc9a334486 admin admin] PATH_INFO: `/v3/projects/f397d95c56cf4ae18500ade3c5d97976/users/e510a33696844d60adbc796dff4102c6/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:04.360184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d9683-9825-44fe-9a5f-74bc9a334486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f397d95c56cf4ae18500ade3c5d97976, user_id=e510a33696844d60adbc796dff4102c6, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:04.363018 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d9683-9825-44fe-9a5f-74bc9a334486 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:04.393511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1133/2257] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:48:04 2022] PUT /identity/v3/projects/f397d95c56cf4ae18500ade3c5d97976/users/e510a33696844d60adbc796dff4102c6/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:48:04.397658 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa0085-1376-44a3-a441-2592d4574609 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:04.397984 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa0085-1376-44a3-a441-2592d4574609 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:04.398199 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa0085-1376-44a3-a441-2592d4574609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:04.455527 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f9aa0085-1376-44a3-a441-2592d4574609 None None] MFA Rules not processed for user `e510a33696844d60adbc796dff4102c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:48:04.457400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f9aa0085-1376-44a3-a441-2592d4574609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:04.482414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:48:04.601128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1125/2258] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:48:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:48:04.603010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d9683-9825-44fe-9a5f-74bc9a334486 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:04.686316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d9683-9825-44fe-9a5f-74bc9a334486 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:04.701764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef2572-26ea-4687-b670-58f564679634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:04.702546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef2572-26ea-4687-b670-58f564679634 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:04.702919 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef2572-26ea-4687-b670-58f564679634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:04.703539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef2572-26ea-4687-b670-58f564679634 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:04.704788 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef2572-26ea-4687-b670-58f564679634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:04.707164 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef2572-26ea-4687-b670-58f564679634 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:04.784199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1134/2259] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:48:04 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:48:04.788075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa0085-1376-44a3-a441-2592d4574609 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:04.805402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa0085-1376-44a3-a441-2592d4574609 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:04.908702 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05529ce-9160-489e-83f3-fcd24a47b0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:04.909369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05529ce-9160-489e-83f3-fcd24a47b0e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:04.909651 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05529ce-9160-489e-83f3-fcd24a47b0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:04.909923 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05529ce-9160-489e-83f3-fcd24a47b0e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:04.910574 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05529ce-9160-489e-83f3-fcd24a47b0e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:04.912898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05529ce-9160-489e-83f3-fcd24a47b0e4 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:04.943625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1126/2260] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:48:04 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:48:04.950086 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef2572-26ea-4687-b670-58f564679634 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:04.969305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef2572-26ea-4687-b670-58f564679634 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:04.983000 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce675320-d386-4e92-bda0-440a39f635f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:04.983713 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce675320-d386-4e92-bda0-440a39f635f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:04.983895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce675320-d386-4e92-bda0-440a39f635f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:04.984169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce675320-d386-4e92-bda0-440a39f635f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:04.985038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce675320-d386-4e92-bda0-440a39f635f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:04.986775 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce675320-d386-4e92-bda0-440a39f635f2 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:05.007164 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1135/2261] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:48:05.008777 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05529ce-9160-489e-83f3-fcd24a47b0e4 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:05.021597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05529ce-9160-489e-83f3-fcd24a47b0e4 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:05.031287 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c280a-b30e-4f85-a371-40ac7b04d4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:05.031943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c280a-b30e-4f85-a371-40ac7b04d4ec admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:05.032291 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c280a-b30e-4f85-a371-40ac7b04d4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:05.032683 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c280a-b30e-4f85-a371-40ac7b04d4ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:05.033608 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c280a-b30e-4f85-a371-40ac7b04d4ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:05.035799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c280a-b30e-4f85-a371-40ac7b04d4ec admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:05.051331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1127/2262] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:48:05.052880 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce675320-d386-4e92-bda0-440a39f635f2 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:05.082928 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce675320-d386-4e92-bda0-440a39f635f2 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:05.100609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0aa2e-aa9f-4eef-a63b-7a05f7d59c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:05.101651 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0aa2e-aa9f-4eef-a63b-7a05f7d59c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:05.102003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0aa2e-aa9f-4eef-a63b-7a05f7d59c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:05.102344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0aa2e-aa9f-4eef-a63b-7a05f7d59c89 admin admin] PATH_INFO: `/v3/projects/5c48eb04e3ba4747b367c7c7e5b7319d/users/49729826127a458bb28ac3db158d694f/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:05.122604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:48:05.134580 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0aa2e-aa9f-4eef-a63b-7a05f7d59c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c48eb04e3ba4747b367c7c7e5b7319d, user_id=49729826127a458bb28ac3db158d694f, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:05.138379 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0aa2e-aa9f-4eef-a63b-7a05f7d59c89 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:05.184115 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1136/2263] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:48:05 2022] PUT /identity/v3/projects/5c48eb04e3ba4747b367c7c7e5b7319d/users/49729826127a458bb28ac3db158d694f/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:48:05.186985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d3367-85e0-4a1f-8336-5d524b37bce7 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:05.187322 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d3367-85e0-4a1f-8336-5d524b37bce7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:05.187643 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d3367-85e0-4a1f-8336-5d524b37bce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:05.241022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8c0d3367-85e0-4a1f-8336-5d524b37bce7 None None] MFA Rules not processed for user `49729826127a458bb28ac3db158d694f`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:48:05.242512 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8c0d3367-85e0-4a1f-8336-5d524b37bce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:05.321664 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1128/2264] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:48:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:48:05.322221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0aa2e-aa9f-4eef-a63b-7a05f7d59c89 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:05.420085 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0aa2e-aa9f-4eef-a63b-7a05f7d59c89 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:05.425430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d72c0-d79f-4a79-9b60-4fb57fce0dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:05.426046 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d72c0-d79f-4a79-9b60-4fb57fce0dfa admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:05.426326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d72c0-d79f-4a79-9b60-4fb57fce0dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:05.426595 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d72c0-d79f-4a79-9b60-4fb57fce0dfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:05.427411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d72c0-d79f-4a79-9b60-4fb57fce0dfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:05.429362 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d72c0-d79f-4a79-9b60-4fb57fce0dfa admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:05.473709 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1137/2265] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:48:05 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:48:05.482372 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d3367-85e0-4a1f-8336-5d524b37bce7 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:05.503747 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d3367-85e0-4a1f-8336-5d524b37bce7 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:05.549687 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:48:05.632967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1a082-ff43-46b2-85a6-0d3c06873409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:05.633867 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1a082-ff43-46b2-85a6-0d3c06873409 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:05.634627 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1a082-ff43-46b2-85a6-0d3c06873409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:05.635172 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1a082-ff43-46b2-85a6-0d3c06873409 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:05.635965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1a082-ff43-46b2-85a6-0d3c06873409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:05.638455 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1a082-ff43-46b2-85a6-0d3c06873409 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:05.671551 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1129/2266] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:48:05 2022] POST /identity/v3/users => generated 299 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:48:05.675670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d72c0-d79f-4a79-9b60-4fb57fce0dfa admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:05.687835 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d72c0-d79f-4a79-9b60-4fb57fce0dfa admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:05.693177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78d03b-19bc-41ca-8dd5-0638c78f70c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:05.694210 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78d03b-19bc-41ca-8dd5-0638c78f70c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:05.694492 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78d03b-19bc-41ca-8dd5-0638c78f70c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:05.695329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78d03b-19bc-41ca-8dd5-0638c78f70c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:05.696013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78d03b-19bc-41ca-8dd5-0638c78f70c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:05.697628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78d03b-19bc-41ca-8dd5-0638c78f70c8 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:05.719422 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1138/2267] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:48:05.725745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1a082-ff43-46b2-85a6-0d3c06873409 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:05.737412 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1a082-ff43-46b2-85a6-0d3c06873409 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:05.742215 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc3623-0e7a-46ce-b32f-e822d9051a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:05.742944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc3623-0e7a-46ce-b32f-e822d9051a09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:05.743223 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc3623-0e7a-46ce-b32f-e822d9051a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:05.743500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc3623-0e7a-46ce-b32f-e822d9051a09 admin admin] PATH_INFO: `/v3/projects/6b430b7eeaf4496d87cdca1a5bd9d535/users/11a892f3d78d4e279c06333d1a9adc6b/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:05.763216 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc3623-0e7a-46ce-b32f-e822d9051a09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b430b7eeaf4496d87cdca1a5bd9d535, user_id=11a892f3d78d4e279c06333d1a9adc6b, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:05.766081 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc3623-0e7a-46ce-b32f-e822d9051a09 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:05.786373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1130/2268] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:48:05 2022] PUT /identity/v3/projects/6b430b7eeaf4496d87cdca1a5bd9d535/users/11a892f3d78d4e279c06333d1a9adc6b/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:48:05.790142 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78d03b-19bc-41ca-8dd5-0638c78f70c8 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:05.802452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78d03b-19bc-41ca-8dd5-0638c78f70c8 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:05.882742 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae2cd1-ce9e-4953-a2c8-c32224b5c685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:05.883459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae2cd1-ce9e-4953-a2c8-c32224b5c685 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:05.883796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae2cd1-ce9e-4953-a2c8-c32224b5c685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:05.884146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae2cd1-ce9e-4953-a2c8-c32224b5c685 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:05.884857 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae2cd1-ce9e-4953-a2c8-c32224b5c685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:05.886592 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae2cd1-ce9e-4953-a2c8-c32224b5c685 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:05.901292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1139/2269] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:48:05.905399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc3623-0e7a-46ce-b32f-e822d9051a09 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:05.916482 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc3623-0e7a-46ce-b32f-e822d9051a09 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:05.923935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896c9df-d8b5-48dd-a720-d84b6e9ef538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:05.924925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896c9df-d8b5-48dd-a720-d84b6e9ef538 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:05.925329 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896c9df-d8b5-48dd-a720-d84b6e9ef538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:05.925697 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896c9df-d8b5-48dd-a720-d84b6e9ef538 admin admin] PATH_INFO: `/v3/projects/6b430b7eeaf4496d87cdca1a5bd9d535/users/11a892f3d78d4e279c06333d1a9adc6b/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:05.927804 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896c9df-d8b5-48dd-a720-d84b6e9ef538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b430b7eeaf4496d87cdca1a5bd9d535, user_id=11a892f3d78d4e279c06333d1a9adc6b, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:05.930628 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896c9df-d8b5-48dd-a720-d84b6e9ef538 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:05.953187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1131/2270] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:48:05 2022] PUT /identity/v3/projects/6b430b7eeaf4496d87cdca1a5bd9d535/users/11a892f3d78d4e279c06333d1a9adc6b/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:48:05.957768 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bffbe-8192-4a5a-8e22-daead0c7bdd2 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:05.958692 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bffbe-8192-4a5a-8e22-daead0c7bdd2 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:05.958919 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bffbe-8192-4a5a-8e22-daead0c7bdd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:05.993074 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-5b2bffbe-8192-4a5a-8e22-daead0c7bdd2 None None] MFA Rules not processed for user `11a892f3d78d4e279c06333d1a9adc6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:48:05.994504 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5b2bffbe-8192-4a5a-8e22-daead0c7bdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:06.068219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:48:06.098760 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1140/2271] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:48:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:48:06.114727 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896c9df-d8b5-48dd-a720-d84b6e9ef538 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:06.116701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9896c9df-d8b5-48dd-a720-d84b6e9ef538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:06.188949 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896c9df-d8b5-48dd-a720-d84b6e9ef538 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:06.193345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d5ed9-eb9a-4023-b8a1-7672d202f406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:06.193958 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d5ed9-eb9a-4023-b8a1-7672d202f406 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:06.194258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d5ed9-eb9a-4023-b8a1-7672d202f406 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:06.194532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d5ed9-eb9a-4023-b8a1-7672d202f406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:06.274198 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d5ed9-eb9a-4023-b8a1-7672d202f406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:06.276313 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d5ed9-eb9a-4023-b8a1-7672d202f406 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:06.287488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1132/2272] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:48:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:48:09.041425 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2bffbe-8192-4a5a-8e22-daead0c7bdd2 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:09.058597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2bffbe-8192-4a5a-8e22-daead0c7bdd2 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:09.063357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917f243-4c74-4f7d-ad8e-972ac8f8d29a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:09.064166 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917f243-4c74-4f7d-ad8e-972ac8f8d29a service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:09.064621 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917f243-4c74-4f7d-ad8e-972ac8f8d29a service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:09.065067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917f243-4c74-4f7d-ad8e-972ac8f8d29a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:09.141074 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917f243-4c74-4f7d-ad8e-972ac8f8d29a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:09.142693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917f243-4c74-4f7d-ad8e-972ac8f8d29a service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:09.154330 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1141/2273] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:48:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:48:09.929618 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d5ed9-eb9a-4023-b8a1-7672d202f406 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:09.940835 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d5ed9-eb9a-4023-b8a1-7672d202f406 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:09.946818 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5e017-01d1-4559-b9e7-bb3c68af9a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:09.947446 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5e017-01d1-4559-b9e7-bb3c68af9a64 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:09.947749 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5e017-01d1-4559-b9e7-bb3c68af9a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:09.948139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5e017-01d1-4559-b9e7-bb3c68af9a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:09.959267 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5e017-01d1-4559-b9e7-bb3c68af9a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:09.961462 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5e017-01d1-4559-b9e7-bb3c68af9a64 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:09.984746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1133/2274] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:48:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:48:11.178637 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0af79-905c-49c0-9e4a-d13f0b82fda0 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:11.179514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0af79-905c-49c0-9e4a-d13f0b82fda0 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:11.179514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0af79-905c-49c0-9e4a-d13f0b82fda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:11.205819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-f7a0af79-905c-49c0-9e4a-d13f0b82fda0 None None] MFA Rules not processed for user `3cab2568112f46bca576f17916887aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:48:11.207716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f7a0af79-905c-49c0-9e4a-d13f0b82fda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:11.292550 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1142/2275] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:48:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:48:13.695044 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac4658-d2c8-459c-a0dc-64ce3a18afc1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:13.696598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac4658-d2c8-459c-a0dc-64ce3a18afc1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:13.697438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac4658-d2c8-459c-a0dc-64ce3a18afc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:13.751386 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a9ac4658-d2c8-459c-a0dc-64ce3a18afc1 None None] MFA Rules not processed for user `0216bf67389e4627bf953b7dcc97a88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:48:13.752929 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a9ac4658-d2c8-459c-a0dc-64ce3a18afc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:13.764326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:48:13.841104 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1134/2276] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 26 05:48:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:48:13.862223 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0af79-905c-49c0-9e4a-d13f0b82fda0 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:13.864377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f7a0af79-905c-49c0-9e4a-d13f0b82fda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:14.006647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0af79-905c-49c0-9e4a-d13f0b82fda0 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:14.017562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423c2d8-78d7-4938-9b9c-725e331fbeea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f258aba84d4dd59ebe8de973ac65c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:14.019957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c2d8-78d7-4938-9b9c-725e331fbeea service swift] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:14.021111 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c2d8-78d7-4938-9b9c-725e331fbeea service swift] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:14.021997 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c2d8-78d7-4938-9b9c-725e331fbeea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:14.043848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423c2d8-78d7-4938-9b9c-725e331fbeea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:14.046176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423c2d8-78d7-4938-9b9c-725e331fbeea service swift] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:14.058548 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1143/2277] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 26 05:48:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:48:15.410080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac4658-d2c8-459c-a0dc-64ce3a18afc1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:15.486846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac4658-d2c8-459c-a0dc-64ce3a18afc1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:15.492246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9eebc-7009-45af-b812-9e1ecada8cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:15.493053 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9eebc-7009-45af-b812-9e1ecada8cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:15.493369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9eebc-7009-45af-b812-9e1ecada8cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:15.493755 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9eebc-7009-45af-b812-9e1ecada8cf0 admin admin] PATH_INFO: `/v3/users/e510a33696844d60adbc796dff4102c6` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:15.495221 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9eebc-7009-45af-b812-9e1ecada8cf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e510a33696844d60adbc796dff4102c6)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:15.498080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9eebc-7009-45af-b812-9e1ecada8cf0 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:15.557491 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d6b9eebc-7009-45af-b812-9e1ecada8cf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e510a33696844d60adbc796dff4102c6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:15.569620 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d6b9eebc-7009-45af-b812-9e1ecada8cf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e510a33696844d60adbc796dff4102c6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:15.570363 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d6b9eebc-7009-45af-b812-9e1ecada8cf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e510a33696844d60adbc796dff4102c6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:15.571069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d6b9eebc-7009-45af-b812-9e1ecada8cf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e510a33696844d60adbc796dff4102c6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:15.586887 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d6b9eebc-7009-45af-b812-9e1ecada8cf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e510a33696844d60adbc796dff4102c6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:15.609450 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1135/2278] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:48:15 2022] DELETE /identity/v3/users/e510a33696844d60adbc796dff4102c6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:48:15.615908 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507dca8-1586-48ae-b65a-30f4ea191802 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:15.616463 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507dca8-1586-48ae-b65a-30f4ea191802 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:15.616463 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507dca8-1586-48ae-b65a-30f4ea191802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:15.638273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-6507dca8-1586-48ae-b65a-30f4ea191802 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:48:15.639639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6507dca8-1586-48ae-b65a-30f4ea191802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:15.659862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:48:15.718778 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1144/2279] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:48:15.729839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9eebc-7009-45af-b812-9e1ecada8cf0 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:15.731659 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d6b9eebc-7009-45af-b812-9e1ecada8cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:15.785164 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:48:15.805044 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9eebc-7009-45af-b812-9e1ecada8cf0 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:15.809498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b5f8c-faad-4331-8f4a-13e17bc8daa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:15.810148 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b5f8c-faad-4331-8f4a-13e17bc8daa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:15.810427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b5f8c-faad-4331-8f4a-13e17bc8daa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:15.810700 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b5f8c-faad-4331-8f4a-13e17bc8daa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:15.822788 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b5f8c-faad-4331-8f4a-13e17bc8daa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:15.824382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b5f8c-faad-4331-8f4a-13e17bc8daa6 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:15.836312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1136/2280] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:48:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:48:16.269483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507dca8-1586-48ae-b65a-30f4ea191802 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:16.271110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6507dca8-1586-48ae-b65a-30f4ea191802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:16.285298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507dca8-1586-48ae-b65a-30f4ea191802 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:16.291664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992c387-4424-48cc-b570-2101cf7cdb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:16.292336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992c387-4424-48cc-b570-2101cf7cdb1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:16.292661 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992c387-4424-48cc-b570-2101cf7cdb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:16.292944 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992c387-4424-48cc-b570-2101cf7cdb1f admin admin] PATH_INFO: `/v3/projects/f397d95c56cf4ae18500ade3c5d97976` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:16.294217 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992c387-4424-48cc-b570-2101cf7cdb1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f397d95c56cf4ae18500ade3c5d97976)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:16.296153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992c387-4424-48cc-b570-2101cf7cdb1f admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:16.357111 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c992c387-4424-48cc-b570-2101cf7cdb1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f397d95c56cf4ae18500ade3c5d97976', 'request_id': 'req-c992c387-4424-48cc-b570-2101cf7cdb1f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:16.364624 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c992c387-4424-48cc-b570-2101cf7cdb1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f397d95c56cf4ae18500ade3c5d97976', 'request_id': 'req-c992c387-4424-48cc-b570-2101cf7cdb1f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:16.384379 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c992c387-4424-48cc-b570-2101cf7cdb1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f397d95c56cf4ae18500ade3c5d97976', 'request_id': 'req-c992c387-4424-48cc-b570-2101cf7cdb1f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:16.399050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c992c387-4424-48cc-b570-2101cf7cdb1f admin admin] The token cache is being invalidate because project f397d95c56cf4ae18500ade3c5d97976 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:48:16.399363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c992c387-4424-48cc-b570-2101cf7cdb1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:16.400017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c992c387-4424-48cc-b570-2101cf7cdb1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:16.401604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1145/2281] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:48:16 2022] DELETE /identity/v3/projects/f397d95c56cf4ae18500ade3c5d97976 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:48:16.406009 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b5f8c-faad-4331-8f4a-13e17bc8daa6 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:16.421581 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b5f8c-faad-4331-8f4a-13e17bc8daa6 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:16.500197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62b516-ca94-4176-a004-85719fdcded2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:16.501461 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62b516-ca94-4176-a004-85719fdcded2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:16.501962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62b516-ca94-4176-a004-85719fdcded2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:16.502429 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62b516-ca94-4176-a004-85719fdcded2 admin admin] PATH_INFO: `/v3/users/49729826127a458bb28ac3db158d694f` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:16.504544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62b516-ca94-4176-a004-85719fdcded2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49729826127a458bb28ac3db158d694f)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:16.508474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62b516-ca94-4176-a004-85719fdcded2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:16.566169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-da62b516-ca94-4176-a004-85719fdcded2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49729826127a458bb28ac3db158d694f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:16.579362 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-da62b516-ca94-4176-a004-85719fdcded2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49729826127a458bb28ac3db158d694f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:16.581281 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-da62b516-ca94-4176-a004-85719fdcded2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49729826127a458bb28ac3db158d694f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:16.582432 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-da62b516-ca94-4176-a004-85719fdcded2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49729826127a458bb28ac3db158d694f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:16.603629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-da62b516-ca94-4176-a004-85719fdcded2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49729826127a458bb28ac3db158d694f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:16.618334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1137/2282] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:48:16 2022] DELETE /identity/v3/users/49729826127a458bb28ac3db158d694f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:48:17.093978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992c387-4424-48cc-b570-2101cf7cdb1f admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:17.095640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c992c387-4424-48cc-b570-2101cf7cdb1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:17.109885 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992c387-4424-48cc-b570-2101cf7cdb1f admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:17.217390 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:17.218195 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:17.218506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:17.218739 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a admin admin] PATH_INFO: `/v3/projects/5c48eb04e3ba4747b367c7c7e5b7319d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:17.219939 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c48eb04e3ba4747b367c7c7e5b7319d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:17.221889 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:17.298888 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:48:17.307541 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c48eb04e3ba4747b367c7c7e5b7319d', 'request_id': 'req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:17.319211 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c48eb04e3ba4747b367c7c7e5b7319d', 'request_id': 'req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:17.367517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c48eb04e3ba4747b367c7c7e5b7319d', 'request_id': 'req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:17.384059 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a admin admin] The token cache is being invalidate because project 5c48eb04e3ba4747b367c7c7e5b7319d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:48:17.384059 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:17.384433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:17.385951 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1146/2283] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:48:17 2022] DELETE /identity/v3/projects/5c48eb04e3ba4747b367c7c7e5b7319d => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:48:17.390320 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62b516-ca94-4176-a004-85719fdcded2 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:17.391788 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-da62b516-ca94-4176-a004-85719fdcded2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:17.406794 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62b516-ca94-4176-a004-85719fdcded2 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:17.502533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f89ab-7654-43f5-a2e0-3400c6003dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:17.503497 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f89ab-7654-43f5-a2e0-3400c6003dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:17.503761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f89ab-7654-43f5-a2e0-3400c6003dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:17.504099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f89ab-7654-43f5-a2e0-3400c6003dfa admin admin] PATH_INFO: `/v3/users/11a892f3d78d4e279c06333d1a9adc6b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:17.505654 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f89ab-7654-43f5-a2e0-3400c6003dfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=11a892f3d78d4e279c06333d1a9adc6b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:17.507770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f89ab-7654-43f5-a2e0-3400c6003dfa admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:17.509708 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:48:17.579152 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-997f89ab-7654-43f5-a2e0-3400c6003dfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11a892f3d78d4e279c06333d1a9adc6b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:17.589305 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-997f89ab-7654-43f5-a2e0-3400c6003dfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11a892f3d78d4e279c06333d1a9adc6b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:17.589844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-997f89ab-7654-43f5-a2e0-3400c6003dfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11a892f3d78d4e279c06333d1a9adc6b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:17.590126 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-997f89ab-7654-43f5-a2e0-3400c6003dfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11a892f3d78d4e279c06333d1a9adc6b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:17.609842 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-997f89ab-7654-43f5-a2e0-3400c6003dfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11a892f3d78d4e279c06333d1a9adc6b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:17.626190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1138/2284] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:48:17 2022] DELETE /identity/v3/users/11a892f3d78d4e279c06333d1a9adc6b => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:48:17.954263 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:17.955795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:17.977660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e13b4-86eb-4ab0-a8c8-aa3bbcfeda6a admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:18.068419 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027d93e-df23-4d7b-a92c-306eb637195e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:18.069158 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027d93e-df23-4d7b-a92c-306eb637195e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:18.069508 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027d93e-df23-4d7b-a92c-306eb637195e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:18.070284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027d93e-df23-4d7b-a92c-306eb637195e admin admin] PATH_INFO: `/v3/projects/6b430b7eeaf4496d87cdca1a5bd9d535` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:18.074499 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027d93e-df23-4d7b-a92c-306eb637195e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b430b7eeaf4496d87cdca1a5bd9d535)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:18.076697 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027d93e-df23-4d7b-a92c-306eb637195e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:18.189233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7027d93e-df23-4d7b-a92c-306eb637195e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b430b7eeaf4496d87cdca1a5bd9d535', 'request_id': 'req-7027d93e-df23-4d7b-a92c-306eb637195e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:18.197051 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7027d93e-df23-4d7b-a92c-306eb637195e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b430b7eeaf4496d87cdca1a5bd9d535', 'request_id': 'req-7027d93e-df23-4d7b-a92c-306eb637195e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:18.208093 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7027d93e-df23-4d7b-a92c-306eb637195e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b430b7eeaf4496d87cdca1a5bd9d535', 'request_id': 'req-7027d93e-df23-4d7b-a92c-306eb637195e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:18.246281 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7027d93e-df23-4d7b-a92c-306eb637195e admin admin] The token cache is being invalidate because project 6b430b7eeaf4496d87cdca1a5bd9d535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:48:18.246927 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7027d93e-df23-4d7b-a92c-306eb637195e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:18.248219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7027d93e-df23-4d7b-a92c-306eb637195e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:18.258125 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1147/2285] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:48:17 2022] DELETE /identity/v3/projects/6b430b7eeaf4496d87cdca1a5bd9d535 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:48:18.259219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b443bdf-64be-4d9a-a190-5f02599083aa None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:18.259498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b443bdf-64be-4d9a-a190-5f02599083aa None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:18.259876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b443bdf-64be-4d9a-a190-5f02599083aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:18.290790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-3b443bdf-64be-4d9a-a190-5f02599083aa None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:48:18.292020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3b443bdf-64be-4d9a-a190-5f02599083aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:18.396838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1139/2286] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:48:18.405114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb1637-dd54-44a5-a5da-d88f7babecc1 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:18.405477 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb1637-dd54-44a5-a5da-d88f7babecc1 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:18.405730 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb1637-dd54-44a5-a5da-d88f7babecc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:18.428987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-f0cb1637-dd54-44a5-a5da-d88f7babecc1 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:48:18.430257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f0cb1637-dd54-44a5-a5da-d88f7babecc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:18.438962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1148/2287] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:48:18.443498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b443bdf-64be-4d9a-a190-5f02599083aa None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:18.454237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b443bdf-64be-4d9a-a190-5f02599083aa None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:18.460178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ed18d-ea5f-4f01-8273-d13361d806ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:18.461543 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ed18d-ea5f-4f01-8273-d13361d806ed admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:18.461924 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ed18d-ea5f-4f01-8273-d13361d806ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:18.462200 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ed18d-ea5f-4f01-8273-d13361d806ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:18.463104 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ed18d-ea5f-4f01-8273-d13361d806ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:48:18.463470 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ed18d-ea5f-4f01-8273-d13361d806ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:18.465267 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ed18d-ea5f-4f01-8273-d13361d806ed admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:18.479229 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1140/2288] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:48:18.484113 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb1637-dd54-44a5-a5da-d88f7babecc1 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:18.494473 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb1637-dd54-44a5-a5da-d88f7babecc1 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:18.499441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419756d-3b05-4a13-ac11-7933ec88b352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:18.500060 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419756d-3b05-4a13-ac11-7933ec88b352 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:18.500336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419756d-3b05-4a13-ac11-7933ec88b352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:18.500632 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419756d-3b05-4a13-ac11-7933ec88b352 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:18.501297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419756d-3b05-4a13-ac11-7933ec88b352 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:18.503663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419756d-3b05-4a13-ac11-7933ec88b352 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:18.535321 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1149/2289] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:48:18 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:48:18.539873 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ed18d-ea5f-4f01-8273-d13361d806ed admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:18.559552 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ed18d-ea5f-4f01-8273-d13361d806ed admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:18.622030 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbf0cc-4aea-44c1-a7c5-1b5896d1a204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:18.622669 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbf0cc-4aea-44c1-a7c5-1b5896d1a204 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:18.622953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbf0cc-4aea-44c1-a7c5-1b5896d1a204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:18.623230 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbf0cc-4aea-44c1-a7c5-1b5896d1a204 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:18.623884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dbf0cc-4aea-44c1-a7c5-1b5896d1a204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:18.625884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dbf0cc-4aea-44c1-a7c5-1b5896d1a204 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:18.639618 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:48:18.658065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1141/2290] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:48:18 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:48:18.662719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419756d-3b05-4a13-ac11-7933ec88b352 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:18.677081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419756d-3b05-4a13-ac11-7933ec88b352 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:18.684975 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e231e-623f-4516-86e3-1ccaff749fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:18.684975 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e231e-623f-4516-86e3-1ccaff749fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:18.684975 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e231e-623f-4516-86e3-1ccaff749fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:18.684975 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e231e-623f-4516-86e3-1ccaff749fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:18.685528 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e231e-623f-4516-86e3-1ccaff749fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:18.687104 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e231e-623f-4516-86e3-1ccaff749fe1 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:18.700576 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1150/2291] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:48:18.704431 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbf0cc-4aea-44c1-a7c5-1b5896d1a204 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:18.721616 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbf0cc-4aea-44c1-a7c5-1b5896d1a204 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:18.727603 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572785a-0cb0-43fb-a817-06dad3041103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:18.728228 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572785a-0cb0-43fb-a817-06dad3041103 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:18.728500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572785a-0cb0-43fb-a817-06dad3041103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:18.728791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572785a-0cb0-43fb-a817-06dad3041103 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:18.729477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572785a-0cb0-43fb-a817-06dad3041103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:18.731116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572785a-0cb0-43fb-a817-06dad3041103 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:18.742904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1142/2292] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:48:18.747577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e231e-623f-4516-86e3-1ccaff749fe1 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:18.758512 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e231e-623f-4516-86e3-1ccaff749fe1 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:18.769188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d2adb-eba4-48bf-86df-a64d5f311389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:18.773284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d2adb-eba4-48bf-86df-a64d5f311389 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:18.773579 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d2adb-eba4-48bf-86df-a64d5f311389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:18.773812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d2adb-eba4-48bf-86df-a64d5f311389 admin admin] PATH_INFO: `/v3/projects/ab233f9ef528422e90eeeb5bee9f3bf3/users/7c783407791148a68e9021b5704cf4a4/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:18.792798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d2adb-eba4-48bf-86df-a64d5f311389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab233f9ef528422e90eeeb5bee9f3bf3, user_id=7c783407791148a68e9021b5704cf4a4, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:18.795525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d2adb-eba4-48bf-86df-a64d5f311389 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:18.807856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:48:18.817099 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1151/2293] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:48:18 2022] PUT /identity/v3/projects/ab233f9ef528422e90eeeb5bee9f3bf3/users/7c783407791148a68e9021b5704cf4a4/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:48:18.827666 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d84d3-1cd2-4286-9db2-f758b2da37b5 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:18.827942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d84d3-1cd2-4286-9db2-f758b2da37b5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:18.828216 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d84d3-1cd2-4286-9db2-f758b2da37b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:18.859938 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-302d84d3-1cd2-4286-9db2-f758b2da37b5 None None] MFA Rules not processed for user `7c783407791148a68e9021b5704cf4a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:48:18.861222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-302d84d3-1cd2-4286-9db2-f758b2da37b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:18.936192 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1143/2294] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:48:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:48:18.948102 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d2adb-eba4-48bf-86df-a64d5f311389 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:18.950045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-1d1d2adb-eba4-48bf-86df-a64d5f311389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:19.020601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d2adb-eba4-48bf-86df-a64d5f311389 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:19.026690 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738f74e-6d15-4402-a521-948cf384e488 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:19.027809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738f74e-6d15-4402-a521-948cf384e488 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:19.028220 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738f74e-6d15-4402-a521-948cf384e488 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:19.028801 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738f74e-6d15-4402-a521-948cf384e488 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:19.038202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738f74e-6d15-4402-a521-948cf384e488 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:19.039687 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738f74e-6d15-4402-a521-948cf384e488 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:19.049677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1152/2295] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:48:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:48:28.932596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=39 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:48:29.045033 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=27 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:48:49.681847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d84d3-1cd2-4286-9db2-f758b2da37b5 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:49.755522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d84d3-1cd2-4286-9db2-f758b2da37b5 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:49.761039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab386d-1c4d-4e64-8199-6b7b7babd4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:49.761748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab386d-1c4d-4e64-8199-6b7b7babd4ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:49.762199 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab386d-1c4d-4e64-8199-6b7b7babd4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:49.762482 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab386d-1c4d-4e64-8199-6b7b7babd4ab admin admin] PATH_INFO: `/v3/users/7c783407791148a68e9021b5704cf4a4` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:49.763814 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab386d-1c4d-4e64-8199-6b7b7babd4ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c783407791148a68e9021b5704cf4a4)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:49.765882 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab386d-1c4d-4e64-8199-6b7b7babd4ab admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:49.819022 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9cab386d-1c4d-4e64-8199-6b7b7babd4ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c783407791148a68e9021b5704cf4a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:49.827351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9cab386d-1c4d-4e64-8199-6b7b7babd4ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c783407791148a68e9021b5704cf4a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:49.827972 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9cab386d-1c4d-4e64-8199-6b7b7babd4ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c783407791148a68e9021b5704cf4a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:49.828587 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9cab386d-1c4d-4e64-8199-6b7b7babd4ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c783407791148a68e9021b5704cf4a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:49.842017 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9cab386d-1c4d-4e64-8199-6b7b7babd4ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c783407791148a68e9021b5704cf4a4'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:49.854642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1144/2296] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:48:49 2022] DELETE /identity/v3/users/7c783407791148a68e9021b5704cf4a4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:48:49.859222 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e24b5-a399-4b56-949e-80858ca565ea None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:49.859765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e24b5-a399-4b56-949e-80858ca565ea None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:49.859765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e24b5-a399-4b56-949e-80858ca565ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:49.879002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-913e24b5-a399-4b56-949e-80858ca565ea None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:48:49.880224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-913e24b5-a399-4b56-949e-80858ca565ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:49.947750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1153/2297] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:48:49.959174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab386d-1c4d-4e64-8199-6b7b7babd4ab admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:49.961599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9cab386d-1c4d-4e64-8199-6b7b7babd4ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:50.042270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab386d-1c4d-4e64-8199-6b7b7babd4ab admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:50.046492 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1134f1-df76-47e9-a740-c0be0f639564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:50.047114 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1134f1-df76-47e9-a740-c0be0f639564 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:50.047390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1134f1-df76-47e9-a740-c0be0f639564 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:50.047657 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1134f1-df76-47e9-a740-c0be0f639564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:50.059152 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1134f1-df76-47e9-a740-c0be0f639564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:50.060708 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1134f1-df76-47e9-a740-c0be0f639564 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:50.071458 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1145/2298] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:48:50.372907 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e24b5-a399-4b56-949e-80858ca565ea None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:48:50.374880 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-913e24b5-a399-4b56-949e-80858ca565ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:48:50.389913 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e24b5-a399-4b56-949e-80858ca565ea None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:48:50.397354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c3d6f-5524-4a09-8359-c00de0e58e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:48:50.398040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c3d6f-5524-4a09-8359-c00de0e58e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:48:50.398320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c3d6f-5524-4a09-8359-c00de0e58e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:48:50.398591 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c3d6f-5524-4a09-8359-c00de0e58e4f admin admin] PATH_INFO: `/v3/projects/ab233f9ef528422e90eeeb5bee9f3bf3` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:48:50.399950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c3d6f-5524-4a09-8359-c00de0e58e4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab233f9ef528422e90eeeb5bee9f3bf3)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:48:50.402176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c3d6f-5524-4a09-8359-c00de0e58e4f admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:48:50.470760 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d51c3d6f-5524-4a09-8359-c00de0e58e4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab233f9ef528422e90eeeb5bee9f3bf3', 'request_id': 'req-d51c3d6f-5524-4a09-8359-c00de0e58e4f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:50.478845 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d51c3d6f-5524-4a09-8359-c00de0e58e4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab233f9ef528422e90eeeb5bee9f3bf3', 'request_id': 'req-d51c3d6f-5524-4a09-8359-c00de0e58e4f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:50.488147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d51c3d6f-5524-4a09-8359-c00de0e58e4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab233f9ef528422e90eeeb5bee9f3bf3', 'request_id': 'req-d51c3d6f-5524-4a09-8359-c00de0e58e4f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:50.503948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d51c3d6f-5524-4a09-8359-c00de0e58e4f admin admin] The token cache is being invalidate because project ab233f9ef528422e90eeeb5bee9f3bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:48:50.505103 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d51c3d6f-5524-4a09-8359-c00de0e58e4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:50.505525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d51c3d6f-5524-4a09-8359-c00de0e58e4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:48:50.507892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1154/2299] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:48:50 2022] DELETE /identity/v3/projects/ab233f9ef528422e90eeeb5bee9f3bf3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:49:00.066763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:49:00.499587 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:49:05.369856 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1134f1-df76-47e9-a740-c0be0f639564 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:49:05.371506 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3b1134f1-df76-47e9-a740-c0be0f639564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:05.384521 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1134f1-df76-47e9-a740-c0be0f639564 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:49:05.448321 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07b1e6-50bc-4e7d-b2be-7cdfce407cc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:49:05.448966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07b1e6-50bc-4e7d-b2be-7cdfce407cc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:05.449307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07b1e6-50bc-4e7d-b2be-7cdfce407cc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:05.449579 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07b1e6-50bc-4e7d-b2be-7cdfce407cc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:05.451591 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bc07b1e6-50bc-4e7d-b2be-7cdfce407cc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:05.524322 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07b1e6-50bc-4e7d-b2be-7cdfce407cc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:49:05.525837 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07b1e6-50bc-4e7d-b2be-7cdfce407cc7 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:49:05.537090 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1146/2300] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:49:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:49:06.971050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c3d6f-5524-4a09-8359-c00de0e58e4f admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:49:06.973309 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d51c3d6f-5524-4a09-8359-c00de0e58e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:06.987014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c3d6f-5524-4a09-8359-c00de0e58e4f admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:49:06.990827 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ba0f4-1092-4fda-8580-372afd7cf8d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:49:06.991508 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ba0f4-1092-4fda-8580-372afd7cf8d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:06.991853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ba0f4-1092-4fda-8580-372afd7cf8d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:06.992185 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ba0f4-1092-4fda-8580-372afd7cf8d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:06.994137 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d68ba0f4-1092-4fda-8580-372afd7cf8d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:07.108966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ba0f4-1092-4fda-8580-372afd7cf8d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:49:07.111841 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ba0f4-1092-4fda-8580-372afd7cf8d8 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:49:07.133774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1155/2301] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:49:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:49:07.254849 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd5c5d-3ec7-4599-b468-cb67d519ed8b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:07.255366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd5c5d-3ec7-4599-b468-cb67d519ed8b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:07.255554 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd5c5d-3ec7-4599-b468-cb67d519ed8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:07.278686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-6afd5c5d-3ec7-4599-b468-cb67d519ed8b None None] MFA Rules not processed for user `0216bf67389e4627bf953b7dcc97a88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:49:07.280037 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6afd5c5d-3ec7-4599-b468-cb67d519ed8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:07.379382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1147/2302] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 26 05:49:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:49:07.391874 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ba0f4-1092-4fda-8580-372afd7cf8d8 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:49:07.393370 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d68ba0f4-1092-4fda-8580-372afd7cf8d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:07.479479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ba0f4-1092-4fda-8580-372afd7cf8d8 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:49:07.487972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115d00e-da2f-454b-8d08-d8539b90ea18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f258aba84d4dd59ebe8de973ac65c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:49:07.488942 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115d00e-da2f-454b-8d08-d8539b90ea18 service swift] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:07.489447 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115d00e-da2f-454b-8d08-d8539b90ea18 service swift] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:07.489889 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115d00e-da2f-454b-8d08-d8539b90ea18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:07.501646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115d00e-da2f-454b-8d08-d8539b90ea18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:49:07.503283 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115d00e-da2f-454b-8d08-d8539b90ea18 service swift] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:49:07.512360 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1156/2303] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 26 05:49:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:49:09.489831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd5c5d-3ec7-4599-b468-cb67d519ed8b None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:49:09.492393 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6afd5c5d-3ec7-4599-b468-cb67d519ed8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:09.514460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd5c5d-3ec7-4599-b468-cb67d519ed8b None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:49:09.519290 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3b246-7817-4edc-a846-824877faf79d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:49:09.520182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3b246-7817-4edc-a846-824877faf79d service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:09.520526 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3b246-7817-4edc-a846-824877faf79d service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:09.521039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3b246-7817-4edc-a846-824877faf79d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:09.528599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-15c3b246-7817-4edc-a846-824877faf79d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:09.545380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3b246-7817-4edc-a846-824877faf79d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:49:09.547158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3b246-7817-4edc-a846-824877faf79d service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:49:09.562892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1148/2304] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:49:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:49:13.967078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90078d9-9b18-405c-865f-79701d11b827 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:13.967701 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90078d9-9b18-405c-865f-79701d11b827 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:13.967701 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90078d9-9b18-405c-865f-79701d11b827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:13.990415 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-b90078d9-9b18-405c-865f-79701d11b827 None None] MFA Rules not processed for user `bfaf6a2b8a2348b4aa603cda5fe624f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:49:13.991699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-b90078d9-9b18-405c-865f-79701d11b827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:14.078221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1157/2305] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:49:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:49:19.557936 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=81 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:49:24.069674 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=91 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:49:26.649730 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3b246-7817-4edc-a846-824877faf79d service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:49:26.651605 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-15c3b246-7817-4edc-a846-824877faf79d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:26.668578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3b246-7817-4edc-a846-824877faf79d service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:49:26.674511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9d137-ba71-419c-b567-5fcc5f2b6e88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:49:26.675125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9d137-ba71-419c-b567-5fcc5f2b6e88 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:26.675413 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9d137-ba71-419c-b567-5fcc5f2b6e88 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:26.675676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9d137-ba71-419c-b567-5fcc5f2b6e88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:26.677540 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b1a9d137-ba71-419c-b567-5fcc5f2b6e88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:26.690141 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a9d137-ba71-419c-b567-5fcc5f2b6e88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:49:26.692327 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a9d137-ba71-419c-b567-5fcc5f2b6e88 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:49:26.704928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1149/2306] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:49:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:49:26.977038 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90078d9-9b18-405c-865f-79701d11b827 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:49:26.978760 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-b90078d9-9b18-405c-865f-79701d11b827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:26.996395 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90078d9-9b18-405c-865f-79701d11b827 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:49:27.003039 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd8c14-5962-4496-82d0-88e766d28592 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:49:27.003659 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd8c14-5962-4496-82d0-88e766d28592 service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:27.003940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd8c14-5962-4496-82d0-88e766d28592 service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:27.004213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd8c14-5962-4496-82d0-88e766d28592 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:27.006093 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-41bd8c14-5962-4496-82d0-88e766d28592 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:27.071556 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd8c14-5962-4496-82d0-88e766d28592 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:49:27.073497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd8c14-5962-4496-82d0-88e766d28592 service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:49:27.083862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1158/2307] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:49:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:49:31.430913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9d137-ba71-419c-b567-5fcc5f2b6e88 service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:49:31.443120 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a9d137-ba71-419c-b567-5fcc5f2b6e88 service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:49:31.454345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977f609-c61a-418a-b9ac-f92393e4292a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:49:31.455168 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977f609-c61a-418a-b9ac-f92393e4292a service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:31.455593 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977f609-c61a-418a-b9ac-f92393e4292a service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:31.455960 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977f609-c61a-418a-b9ac-f92393e4292a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:31.458578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9977f609-c61a-418a-b9ac-f92393e4292a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:31.470229 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977f609-c61a-418a-b9ac-f92393e4292a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:49:31.471576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977f609-c61a-418a-b9ac-f92393e4292a service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:49:31.479634 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1150/2308] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:49:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:49:31.619320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c733c3-13a0-404c-b82a-90ae86ef2fb9 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:31.619948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c733c3-13a0-404c-b82a-90ae86ef2fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:31.619948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c733c3-13a0-404c-b82a-90ae86ef2fb9 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:31.621577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1159/2309] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 26 05:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:49:31.628266 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93911552-a8ac-4b62-8297-aaa239944221 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:31.628679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93911552-a8ac-4b62-8297-aaa239944221 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:31.628981 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93911552-a8ac-4b62-8297-aaa239944221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:31.651705 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-93911552-a8ac-4b62-8297-aaa239944221 None None] MFA Rules not processed for user `92556048cbd04c409c3f4b6fd645627f`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:49:31.652950 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-93911552-a8ac-4b62-8297-aaa239944221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:31.663818 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1151/2310] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 26 05:49:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:49:31.676136 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c733c3-13a0-404c-b82a-90ae86ef2fb9 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:49:31.677905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-98c733c3-13a0-404c-b82a-90ae86ef2fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:31.745991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c733c3-13a0-404c-b82a-90ae86ef2fb9 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:49:31.749766 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6b883-2ce2-4370-a39e-05cc1d50ef6b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c0fabecff2457e931c8c1c36681197', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:49:31.750399 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6b883-2ce2-4370-a39e-05cc1d50ef6b service glance] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:31.750679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6b883-2ce2-4370-a39e-05cc1d50ef6b service glance] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:31.750958 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6b883-2ce2-4370-a39e-05cc1d50ef6b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:31.762567 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad6b883-2ce2-4370-a39e-05cc1d50ef6b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:49:31.764023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad6b883-2ce2-4370-a39e-05cc1d50ef6b service glance] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:49:31.775532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1160/2311] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:49:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:49:31.862982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c110c-bbfe-4e3d-8dcb-27ad4e34999f None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:31.863221 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c110c-bbfe-4e3d-8dcb-27ad4e34999f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:31.863533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c110c-bbfe-4e3d-8dcb-27ad4e34999f None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:31.864913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1152/2312] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 26 05:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:49:31.870743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c588a95-82b0-47fc-b240-e323c8111e78 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:31.871046 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c588a95-82b0-47fc-b240-e323c8111e78 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:31.871402 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c588a95-82b0-47fc-b240-e323c8111e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:31.895283 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-6c588a95-82b0-47fc-b240-e323c8111e78 None None] MFA Rules not processed for user `6bf0f594ce7646b7b6ddb8a8553509d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:49:31.896497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6c588a95-82b0-47fc-b240-e323c8111e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:31.904475 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1161/2313] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 26 05:49:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:49:31.919093 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c110c-bbfe-4e3d-8dcb-27ad4e34999f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:49:31.943356 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c110c-bbfe-4e3d-8dcb-27ad4e34999f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:49:31.948169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a7e86-3913-4c25-8afc-ca7e76ce4600 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:49:31.948776 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a7e86-3913-4c25-8afc-ca7e76ce4600 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:31.949056 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a7e86-3913-4c25-8afc-ca7e76ce4600 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:31.949362 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a7e86-3913-4c25-8afc-ca7e76ce4600 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:31.957008 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a7e86-3913-4c25-8afc-ca7e76ce4600 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:49:31.958486 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a7e86-3913-4c25-8afc-ca7e76ce4600 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:49:31.966946 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1153/2314] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:49:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:49:32.053620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b214530-f8cf-494e-9240-826158602f37 None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:32.053907 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b214530-f8cf-494e-9240-826158602f37 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:32.054275 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b214530-f8cf-494e-9240-826158602f37 None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:32.055630 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1162/2315] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 26 05:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:49:32.061323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac104a6f-1a69-4a9d-951f-7c107d35f3a4 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:32.061603 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac104a6f-1a69-4a9d-951f-7c107d35f3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:32.061869 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac104a6f-1a69-4a9d-951f-7c107d35f3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:32.082319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ac104a6f-1a69-4a9d-951f-7c107d35f3a4 None None] MFA Rules not processed for user `92556048cbd04c409c3f4b6fd645627f`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:49:32.083514 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ac104a6f-1a69-4a9d-951f-7c107d35f3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:32.094000 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1154/2316] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 26 05:49:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:49:32.104835 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b214530-f8cf-494e-9240-826158602f37 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:49:32.116347 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b214530-f8cf-494e-9240-826158602f37 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:49:32.120447 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f2b67-2e78-4ba0-9a82-f4a85339a1c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:49:32.121049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f2b67-2e78-4ba0-9a82-f4a85339a1c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:32.121370 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f2b67-2e78-4ba0-9a82-f4a85339a1c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:32.121626 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f2b67-2e78-4ba0-9a82-f4a85339a1c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:32.133127 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f2b67-2e78-4ba0-9a82-f4a85339a1c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:49:32.134734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f2b67-2e78-4ba0-9a82-f4a85339a1c7 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:49:32.145495 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1163/2317] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:49:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:49:32.215798 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac104a6f-1a69-4a9d-951f-7c107d35f3a4 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:49:32.217534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ac104a6f-1a69-4a9d-951f-7c107d35f3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:32.290305 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac104a6f-1a69-4a9d-951f-7c107d35f3a4 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:49:32.295606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9442bdd-8163-4b6b-a8c1-90e65803f89f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:49:32.296213 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9442bdd-8163-4b6b-a8c1-90e65803f89f service placement] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:32.296495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9442bdd-8163-4b6b-a8c1-90e65803f89f service placement] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:32.296754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9442bdd-8163-4b6b-a8c1-90e65803f89f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:32.298580 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b9442bdd-8163-4b6b-a8c1-90e65803f89f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:32.310201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9442bdd-8163-4b6b-a8c1-90e65803f89f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:49:32.311790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9442bdd-8163-4b6b-a8c1-90e65803f89f service placement] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:49:32.321962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1155/2318] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:49:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:49:34.443451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f2b67-2e78-4ba0-9a82-f4a85339a1c7 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:49:34.457474 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f2b67-2e78-4ba0-9a82-f4a85339a1c7 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:49:34.463325 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea642c-b034-4510-8ca9-33de692785d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:49:34.463943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea642c-b034-4510-8ca9-33de692785d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:34.464224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea642c-b034-4510-8ca9-33de692785d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:34.464479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea642c-b034-4510-8ca9-33de692785d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:34.466670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-67ea642c-b034-4510-8ca9-33de692785d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:34.476743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea642c-b034-4510-8ca9-33de692785d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:49:34.478184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea642c-b034-4510-8ca9-33de692785d2 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:49:34.486274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1164/2319] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:49:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:49:35.694350 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f42c2-e789-4566-8c0a-e7435acf6db4 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:35.694828 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f42c2-e789-4566-8c0a-e7435acf6db4 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:35.694940 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f42c2-e789-4566-8c0a-e7435acf6db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:35.715672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-bc6f42c2-e789-4566-8c0a-e7435acf6db4 None None] MFA Rules not processed for user `0216bf67389e4627bf953b7dcc97a88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:49:35.716868 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-bc6f42c2-e789-4566-8c0a-e7435acf6db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:49:35.725186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1156/2320] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 26 05:49:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:49:35.737423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea642c-b034-4510-8ca9-33de692785d2 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:49:35.750781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea642c-b034-4510-8ca9-33de692785d2 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:49:35.758222 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1301ab1-2683-48ce-b973-8b914c3b4118 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f258aba84d4dd59ebe8de973ac65c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:49:35.758812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1301ab1-2683-48ce-b973-8b914c3b4118 service swift] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:49:35.759118 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1301ab1-2683-48ce-b973-8b914c3b4118 service swift] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:49:35.759374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1301ab1-2683-48ce-b973-8b914c3b4118 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:49:35.768461 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1301ab1-2683-48ce-b973-8b914c3b4118 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:49:35.770292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1301ab1-2683-48ce-b973-8b914c3b4118 service swift] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:49:35.778949 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1165/2321] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 26 05:49:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:49:45.709351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=59 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:49:45.775368 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=75 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:50:06.074269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f42c2-e789-4566-8c0a-e7435acf6db4 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:50:06.088718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f42c2-e789-4566-8c0a-e7435acf6db4 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:50:06.095232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c87cd-a30e-435d-99fd-2ae62cae79b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:50:06.095853 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c87cd-a30e-435d-99fd-2ae62cae79b2 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:50:06.096129 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c87cd-a30e-435d-99fd-2ae62cae79b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:50:06.096402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c87cd-a30e-435d-99fd-2ae62cae79b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:50:06.098400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-da0c87cd-a30e-435d-99fd-2ae62cae79b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:50:06.158633 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c87cd-a30e-435d-99fd-2ae62cae79b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:50:06.160254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c87cd-a30e-435d-99fd-2ae62cae79b2 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:50:06.170286 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1157/2322] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:50:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:50:06.785643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e5fe8-8866-4109-bfde-d8bf8a773997 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:50:06.785643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e5fe8-8866-4109-bfde-d8bf8a773997 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:50:06.786971 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e5fe8-8866-4109-bfde-d8bf8a773997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:50:06.809116 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-492e5fe8-8866-4109-bfde-d8bf8a773997 None None] MFA Rules not processed for user `0216bf67389e4627bf953b7dcc97a88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:50:06.809787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-492e5fe8-8866-4109-bfde-d8bf8a773997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:50:06.819293 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1166/2323] 10.222.0.57 () {58 vars in 967 bytes} [Mon Sep 26 05:50:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:50:06.839653 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c87cd-a30e-435d-99fd-2ae62cae79b2 service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:50:06.852484 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c87cd-a30e-435d-99fd-2ae62cae79b2 service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:50:06.857946 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2f4fe-3a56-4d90-a412-498c5cda7929 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f258aba84d4dd59ebe8de973ac65c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:50:06.858553 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2f4fe-3a56-4d90-a412-498c5cda7929 service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:50:06.858824 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2f4fe-3a56-4d90-a412-498c5cda7929 service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:50:06.859083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2f4fe-3a56-4d90-a412-498c5cda7929 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:50:06.867780 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f2f4fe-3a56-4d90-a412-498c5cda7929 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:50:06.869369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f2f4fe-3a56-4d90-a412-498c5cda7929 service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:50:06.879467 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1158/2324] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 26 05:50:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:50:16.803087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:50:16.876700 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=34 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:50:28.052596 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e5fe8-8866-4109-bfde-d8bf8a773997 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:50:28.066692 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e5fe8-8866-4109-bfde-d8bf8a773997 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:50:28.073404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd7488-40e9-4f39-bbbe-99cadc5c9d9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:50:28.073999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd7488-40e9-4f39-bbbe-99cadc5c9d9e service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:50:28.074275 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd7488-40e9-4f39-bbbe-99cadc5c9d9e service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:50:28.074540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd7488-40e9-4f39-bbbe-99cadc5c9d9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:50:28.076346 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-83dd7488-40e9-4f39-bbbe-99cadc5c9d9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:50:28.137841 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd7488-40e9-4f39-bbbe-99cadc5c9d9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:50:28.139348 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd7488-40e9-4f39-bbbe-99cadc5c9d9e service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:50:28.148972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1167/2325] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:50:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:50:36.416131 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2e130-ebc4-404d-b7e6-1ad0c6061f52 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:50:36.416708 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2e130-ebc4-404d-b7e6-1ad0c6061f52 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:50:36.417040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2e130-ebc4-404d-b7e6-1ad0c6061f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:50:36.462713 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-99b2e130-ebc4-404d-b7e6-1ad0c6061f52 None None] MFA Rules not processed for user `342ce12282264d40b2f2bb6f1fc6ba25`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:50:36.464015 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-99b2e130-ebc4-404d-b7e6-1ad0c6061f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:50:36.493354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1159/2326] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:50:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:50:38.144607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=28 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:50:43.177570 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd7488-40e9-4f39-bbbe-99cadc5c9d9e service nova] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:50:43.262429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd7488-40e9-4f39-bbbe-99cadc5c9d9e service nova] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:50:43.269623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffd551-cd9d-40c3-a964-8ace9bb61658 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:50:43.270181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffd551-cd9d-40c3-a964-8ace9bb61658 service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:50:43.270464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffd551-cd9d-40c3-a964-8ace9bb61658 service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:50:43.270754 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffd551-cd9d-40c3-a964-8ace9bb61658 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:50:43.272678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-06ffd551-cd9d-40c3-a964-8ace9bb61658 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:50:43.298582 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ffd551-cd9d-40c3-a964-8ace9bb61658 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:50:43.300628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ffd551-cd9d-40c3-a964-8ace9bb61658 service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:50:43.312263 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1168/2327] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:50:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:50:46.491224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=15 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:50:53.308083 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=24 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:51:33.824516 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2e130-ebc4-404d-b7e6-1ad0c6061f52 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:51:33.840928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2e130-ebc4-404d-b7e6-1ad0c6061f52 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:51:33.857600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee628b-daa5-461c-880c-e40598257887 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:51:33.858454 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee628b-daa5-461c-880c-e40598257887 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:51:33.858939 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee628b-daa5-461c-880c-e40598257887 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:51:33.859380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee628b-daa5-461c-880c-e40598257887 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:51:33.861784 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-21ee628b-daa5-461c-880c-e40598257887 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:51:33.877628 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee628b-daa5-461c-880c-e40598257887 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:51:33.879706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee628b-daa5-461c-880c-e40598257887 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:51:33.901414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1160/2328] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:51:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:51:37.573749 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248d501-e5d2-45a2-a88b-ab6e471c72aa None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:51:37.574387 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248d501-e5d2-45a2-a88b-ab6e471c72aa None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:51:37.574549 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248d501-e5d2-45a2-a88b-ab6e471c72aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:51:37.595795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-1248d501-e5d2-45a2-a88b-ab6e471c72aa None None] MFA Rules not processed for user `9d9e9c8e26774e739695b7e4d42eeafd`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:51:37.597279 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-1248d501-e5d2-45a2-a88b-ab6e471c72aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:51:37.604525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1169/2329] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:51:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:51:37.965161 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee628b-daa5-461c-880c-e40598257887 service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:51:37.989717 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee628b-daa5-461c-880c-e40598257887 service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:51:37.993857 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ee9dc-26dd-4648-ab44-76a37bed05c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:51:37.994569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ee9dc-26dd-4648-ab44-76a37bed05c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:51:37.994883 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ee9dc-26dd-4648-ab44-76a37bed05c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:51:37.995127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ee9dc-26dd-4648-ab44-76a37bed05c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:51:37.997009 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0a8ee9dc-26dd-4648-ab44-76a37bed05c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:51:38.021833 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ee9dc-26dd-4648-ab44-76a37bed05c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:51:38.023654 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ee9dc-26dd-4648-ab44-76a37bed05c2 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:51:38.032815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1161/2330] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:51:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:51:44.481106 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf105e-b8e1-477e-9fc9-3fece0b23783 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:51:44.481671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf105e-b8e1-477e-9fc9-3fece0b23783 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:51:44.481796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf105e-b8e1-477e-9fc9-3fece0b23783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:51:44.501087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-8abf105e-b8e1-477e-9fc9-3fece0b23783 None None] MFA Rules not processed for user `bfaf6a2b8a2348b4aa603cda5fe624f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:51:44.502341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-8abf105e-b8e1-477e-9fc9-3fece0b23783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:51:44.509846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1170/2331] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:51:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:51:48.028654 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=21 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:51:50.358852 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ee9dc-26dd-4648-ab44-76a37bed05c2 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:51:50.381194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ee9dc-26dd-4648-ab44-76a37bed05c2 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:51:50.390911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce73557-c770-46cf-8134-03a830cb21e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:51:50.392354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce73557-c770-46cf-8134-03a830cb21e3 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:51:50.392863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce73557-c770-46cf-8134-03a830cb21e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:51:50.393402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce73557-c770-46cf-8134-03a830cb21e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:51:50.396199 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3ce73557-c770-46cf-8134-03a830cb21e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:51:50.416626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce73557-c770-46cf-8134-03a830cb21e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:51:50.419889 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce73557-c770-46cf-8134-03a830cb21e3 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:51:50.445247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1162/2332] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:51:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:51:54.495749 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=6 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:00.432527 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=6 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:12.669983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c66528-ee66-4b58-ad4c-7fe93ebf1a74 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:12.670553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c66528-ee66-4b58-ad4c-7fe93ebf1a74 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:12.670553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c66528-ee66-4b58-ad4c-7fe93ebf1a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:12.697588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-e1c66528-ee66-4b58-ad4c-7fe93ebf1a74 None None] MFA Rules not processed for user `8c26eec9a5e841f8b760cc8cef309b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:12.698804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-e1c66528-ee66-4b58-ad4c-7fe93ebf1a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:12.706172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1171/2333] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:21.073112 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce73557-c770-46cf-8134-03a830cb21e3 service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:21.075090 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3ce73557-c770-46cf-8134-03a830cb21e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:21.123765 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce73557-c770-46cf-8134-03a830cb21e3 service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:21.131126 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af73671-64af-462d-8480-ff094bf4060e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:21.131677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af73671-64af-462d-8480-ff094bf4060e service placement] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:21.131797 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af73671-64af-462d-8480-ff094bf4060e service placement] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:21.132081 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af73671-64af-462d-8480-ff094bf4060e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:21.134655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0af73671-64af-462d-8480-ff094bf4060e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:21.149717 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af73671-64af-462d-8480-ff094bf4060e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:21.151442 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af73671-64af-462d-8480-ff094bf4060e service placement] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:21.163342 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1163/2334] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:52:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:52:22.323593 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c66528-ee66-4b58-ad4c-7fe93ebf1a74 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:22.332017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c66528-ee66-4b58-ad4c-7fe93ebf1a74 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:22.336115 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a10246-1af3-4d22-941c-2b073a04adc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:22.336977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a10246-1af3-4d22-941c-2b073a04adc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:22.337547 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a10246-1af3-4d22-941c-2b073a04adc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:22.337866 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a10246-1af3-4d22-941c-2b073a04adc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:22.339816 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-b7a10246-1af3-4d22-941c-2b073a04adc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:22.437582 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a10246-1af3-4d22-941c-2b073a04adc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:22.439355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a10246-1af3-4d22-941c-2b073a04adc8 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:22.468062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1172/2335] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:52:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:52:26.878679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af73671-64af-462d-8480-ff094bf4060e service placement] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:26.891552 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af73671-64af-462d-8480-ff094bf4060e service placement] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:26.899257 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aff81d-fc70-4e5c-971e-c4761e915226 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:26.900136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aff81d-fc70-4e5c-971e-c4761e915226 service placement] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:26.900572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aff81d-fc70-4e5c-971e-c4761e915226 service placement] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:26.900999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aff81d-fc70-4e5c-971e-c4761e915226 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:26.903825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-41aff81d-fc70-4e5c-971e-c4761e915226 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:26.918899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aff81d-fc70-4e5c-971e-c4761e915226 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:26.921020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aff81d-fc70-4e5c-971e-c4761e915226 service placement] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:26.955039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1164/2336] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:52:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:52:26.959585 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a10246-1af3-4d22-941c-2b073a04adc8 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:26.962143 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-b7a10246-1af3-4d22-941c-2b073a04adc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:26.977649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a10246-1af3-4d22-941c-2b073a04adc8 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:26.983451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae941e-f88a-43be-ab29-dfbcf436e357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:26.984307 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae941e-f88a-43be-ab29-dfbcf436e357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:26.984755 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae941e-f88a-43be-ab29-dfbcf436e357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:26.985377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae941e-f88a-43be-ab29-dfbcf436e357 admin admin] PATH_INFO: `/v3/users/342ce12282264d40b2f2bb6f1fc6ba25` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:26.986869 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae941e-f88a-43be-ab29-dfbcf436e357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=342ce12282264d40b2f2bb6f1fc6ba25)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:26.989441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae941e-f88a-43be-ab29-dfbcf436e357 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:27.048008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-05ae941e-f88a-43be-ab29-dfbcf436e357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '342ce12282264d40b2f2bb6f1fc6ba25'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:27.056385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-05ae941e-f88a-43be-ab29-dfbcf436e357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '342ce12282264d40b2f2bb6f1fc6ba25'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:27.057012 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-05ae941e-f88a-43be-ab29-dfbcf436e357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '342ce12282264d40b2f2bb6f1fc6ba25'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:27.057810 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-05ae941e-f88a-43be-ab29-dfbcf436e357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '342ce12282264d40b2f2bb6f1fc6ba25'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:27.081123 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-05ae941e-f88a-43be-ab29-dfbcf436e357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '342ce12282264d40b2f2bb6f1fc6ba25'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:27.114392 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1173/2337] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:52:26 2022] DELETE /identity/v3/users/342ce12282264d40b2f2bb6f1fc6ba25 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:27.449269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aff81d-fc70-4e5c-971e-c4761e915226 service placement] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:27.484187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aff81d-fc70-4e5c-971e-c4761e915226 service placement] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:27.562516 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5be76-1be4-44b4-9006-556ea1ac9c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:27.563160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5be76-1be4-44b4-9006-556ea1ac9c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:27.563437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5be76-1be4-44b4-9006-556ea1ac9c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:27.563708 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5be76-1be4-44b4-9006-556ea1ac9c02 admin admin] PATH_INFO: `/v3/projects/c95cba08e61d4d199a3c69d99c5bc77b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:27.564868 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5be76-1be4-44b4-9006-556ea1ac9c02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c95cba08e61d4d199a3c69d99c5bc77b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:27.566914 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5be76-1be4-44b4-9006-556ea1ac9c02 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:27.622958 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9df5be76-1be4-44b4-9006-556ea1ac9c02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c95cba08e61d4d199a3c69d99c5bc77b', 'request_id': 'req-9df5be76-1be4-44b4-9006-556ea1ac9c02'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:27.629501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9df5be76-1be4-44b4-9006-556ea1ac9c02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c95cba08e61d4d199a3c69d99c5bc77b', 'request_id': 'req-9df5be76-1be4-44b4-9006-556ea1ac9c02'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:27.639401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9df5be76-1be4-44b4-9006-556ea1ac9c02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c95cba08e61d4d199a3c69d99c5bc77b', 'request_id': 'req-9df5be76-1be4-44b4-9006-556ea1ac9c02'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:27.650882 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9df5be76-1be4-44b4-9006-556ea1ac9c02 admin admin] The token cache is being invalidate because project c95cba08e61d4d199a3c69d99c5bc77b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:52:27.651178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9df5be76-1be4-44b4-9006-556ea1ac9c02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:27.651876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9df5be76-1be4-44b4-9006-556ea1ac9c02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:27.653765 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1165/2338] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:52:27 2022] DELETE /identity/v3/projects/c95cba08e61d4d199a3c69d99c5bc77b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:27.659240 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d473ef6-6b37-4704-8b66-b7cca1b5dc0e None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:27.659622 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d473ef6-6b37-4704-8b66-b7cca1b5dc0e None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:27.659909 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d473ef6-6b37-4704-8b66-b7cca1b5dc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:27.710793 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:27.733867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-8d473ef6-6b37-4704-8b66-b7cca1b5dc0e None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:27.735113 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-8d473ef6-6b37-4704-8b66-b7cca1b5dc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:27.800295 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1174/2339] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:27.808451 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2592c-476a-470e-88f8-4d3792924b26 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:27.808827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2592c-476a-470e-88f8-4d3792924b26 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:27.809233 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2592c-476a-470e-88f8-4d3792924b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:27.833414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7fd2592c-476a-470e-88f8-4d3792924b26 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:27.834655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7fd2592c-476a-470e-88f8-4d3792924b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:27.845686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1166/2340] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:27.852794 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d473ef6-6b37-4704-8b66-b7cca1b5dc0e None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:27.864620 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d473ef6-6b37-4704-8b66-b7cca1b5dc0e None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:27.870434 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdeeaaa-07af-4a4a-8848-efa8267afe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:27.871063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdeeaaa-07af-4a4a-8848-efa8267afe37 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:27.871338 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdeeaaa-07af-4a4a-8848-efa8267afe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:27.871640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdeeaaa-07af-4a4a-8848-efa8267afe37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:27.872288 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdeeaaa-07af-4a4a-8848-efa8267afe37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:52:27.872588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdeeaaa-07af-4a4a-8848-efa8267afe37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:27.874278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdeeaaa-07af-4a4a-8848-efa8267afe37 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:27.887797 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1175/2341] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:52:27.894087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2592c-476a-470e-88f8-4d3792924b26 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:27.905764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2592c-476a-470e-88f8-4d3792924b26 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:27.914147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afdc3c5-bc9d-4871-b1fd-e42ff66cf89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:27.914798 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdc3c5-bc9d-4871-b1fd-e42ff66cf89f admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:27.915078 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdc3c5-bc9d-4871-b1fd-e42ff66cf89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:27.915347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdc3c5-bc9d-4871-b1fd-e42ff66cf89f admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:27.916217 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afdc3c5-bc9d-4871-b1fd-e42ff66cf89f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:27.919146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afdc3c5-bc9d-4871-b1fd-e42ff66cf89f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:27.959809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1167/2342] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:52:27 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:52:27.965965 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdeeaaa-07af-4a4a-8848-efa8267afe37 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:27.980880 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdeeaaa-07af-4a4a-8848-efa8267afe37 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:28.043258 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04832ea1-ebf3-4298-bb2c-a856d3f94cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:28.043972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04832ea1-ebf3-4298-bb2c-a856d3f94cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:28.044259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04832ea1-ebf3-4298-bb2c-a856d3f94cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:28.044525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04832ea1-ebf3-4298-bb2c-a856d3f94cc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:28.045174 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04832ea1-ebf3-4298-bb2c-a856d3f94cc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:28.047105 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04832ea1-ebf3-4298-bb2c-a856d3f94cc9 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:28.073045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1176/2343] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:52:27 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:52:28.077604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afdc3c5-bc9d-4871-b1fd-e42ff66cf89f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:28.088582 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afdc3c5-bc9d-4871-b1fd-e42ff66cf89f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:28.093993 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1491b-de01-4646-a2d8-e7ae4d823c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:28.094613 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1491b-de01-4646-a2d8-e7ae4d823c08 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:28.094896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1491b-de01-4646-a2d8-e7ae4d823c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:28.095165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1491b-de01-4646-a2d8-e7ae4d823c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:28.095813 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1491b-de01-4646-a2d8-e7ae4d823c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:28.097534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1491b-de01-4646-a2d8-e7ae4d823c08 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:28.099058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:28.112870 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1168/2344] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:52:28.115987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04832ea1-ebf3-4298-bb2c-a856d3f94cc9 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:28.126628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04832ea1-ebf3-4298-bb2c-a856d3f94cc9 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:28.132979 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57021c11-bcc9-441c-bab0-f732a5dacb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:28.133618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57021c11-bcc9-441c-bab0-f732a5dacb1b admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:28.133894 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57021c11-bcc9-441c-bab0-f732a5dacb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:28.134162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57021c11-bcc9-441c-bab0-f732a5dacb1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:28.134805 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57021c11-bcc9-441c-bab0-f732a5dacb1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:28.136427 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57021c11-bcc9-441c-bab0-f732a5dacb1b admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:28.147353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1177/2345] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:52:28.151871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1491b-de01-4646-a2d8-e7ae4d823c08 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:28.162186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1491b-de01-4646-a2d8-e7ae4d823c08 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:28.167126 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f4bcc-ebe1-4332-ad1d-36d2b5b97ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:28.167786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f4bcc-ebe1-4332-ad1d-36d2b5b97ef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:28.168070 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f4bcc-ebe1-4332-ad1d-36d2b5b97ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:28.168341 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f4bcc-ebe1-4332-ad1d-36d2b5b97ef3 admin admin] PATH_INFO: `/v3/projects/15d8f644bc9342379a8c6007f00aa363/users/09f95547d1a04222a7965b83918905aa/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:28.183890 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f4bcc-ebe1-4332-ad1d-36d2b5b97ef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15d8f644bc9342379a8c6007f00aa363, user_id=09f95547d1a04222a7965b83918905aa, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:28.186663 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f4bcc-ebe1-4332-ad1d-36d2b5b97ef3 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:28.202205 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1169/2346] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:52:28 2022] PUT /identity/v3/projects/15d8f644bc9342379a8c6007f00aa363/users/09f95547d1a04222a7965b83918905aa/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:28.214024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edb7cf-e8d7-4abd-a9db-843828d713e7 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:28.216357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edb7cf-e8d7-4abd-a9db-843828d713e7 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:28.216794 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edb7cf-e8d7-4abd-a9db-843828d713e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:28.252101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d5edb7cf-e8d7-4abd-a9db-843828d713e7 None None] MFA Rules not processed for user `09f95547d1a04222a7965b83918905aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:28.253690 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d5edb7cf-e8d7-4abd-a9db-843828d713e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:28.340599 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:28.355774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1178/2347] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:28.362328 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f4bcc-ebe1-4332-ad1d-36d2b5b97ef3 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:28.364302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-979f4bcc-ebe1-4332-ad1d-36d2b5b97ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:28.479899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f4bcc-ebe1-4332-ad1d-36d2b5b97ef3 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:28.486006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35910d74-3236-4934-ac62-e855f33a5160 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:28.486676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35910d74-3236-4934-ac62-e855f33a5160 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:28.487020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35910d74-3236-4934-ac62-e855f33a5160 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:28.487296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35910d74-3236-4934-ac62-e855f33a5160 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:28.495825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35910d74-3236-4934-ac62-e855f33a5160 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:28.497503 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35910d74-3236-4934-ac62-e855f33a5160 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:28.506860 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1170/2348] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:52:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:52:28.731958 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edb7cf-e8d7-4abd-a9db-843828d713e7 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:28.797034 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edb7cf-e8d7-4abd-a9db-843828d713e7 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:28.802073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7bcca-9219-43b6-8c4b-24eb1adcb9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:28.802966 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7bcca-9219-43b6-8c4b-24eb1adcb9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:28.803192 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7bcca-9219-43b6-8c4b-24eb1adcb9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:28.803543 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7bcca-9219-43b6-8c4b-24eb1adcb9fd admin admin] PATH_INFO: `/v3/users/09f95547d1a04222a7965b83918905aa` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:28.804712 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7bcca-9219-43b6-8c4b-24eb1adcb9fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=09f95547d1a04222a7965b83918905aa)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:28.806671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7bcca-9219-43b6-8c4b-24eb1adcb9fd admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:28.869060 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-37b7bcca-9219-43b6-8c4b-24eb1adcb9fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09f95547d1a04222a7965b83918905aa'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:28.877823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-37b7bcca-9219-43b6-8c4b-24eb1adcb9fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09f95547d1a04222a7965b83918905aa'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:28.878539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-37b7bcca-9219-43b6-8c4b-24eb1adcb9fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09f95547d1a04222a7965b83918905aa'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:28.879235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-37b7bcca-9219-43b6-8c4b-24eb1adcb9fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09f95547d1a04222a7965b83918905aa'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:28.887690 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-37b7bcca-9219-43b6-8c4b-24eb1adcb9fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09f95547d1a04222a7965b83918905aa'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:28.911574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1179/2349] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:52:28 2022] DELETE /identity/v3/users/09f95547d1a04222a7965b83918905aa => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:28.916202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee017100-52bb-4da5-a8a4-c089af404c4f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:28.916791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee017100-52bb-4da5-a8a4-c089af404c4f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:28.916791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee017100-52bb-4da5-a8a4-c089af404c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:28.940954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ee017100-52bb-4da5-a8a4-c089af404c4f None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:28.942330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ee017100-52bb-4da5-a8a4-c089af404c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:29.014166 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1171/2350] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:29.024595 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7bcca-9219-43b6-8c4b-24eb1adcb9fd admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:29.026335 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-37b7bcca-9219-43b6-8c4b-24eb1adcb9fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:29.093478 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7bcca-9219-43b6-8c4b-24eb1adcb9fd admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:29.097521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666b3f8-7d85-4c77-989d-ff75a30a5b71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:29.098164 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666b3f8-7d85-4c77-989d-ff75a30a5b71 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:29.098478 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666b3f8-7d85-4c77-989d-ff75a30a5b71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:29.098753 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666b3f8-7d85-4c77-989d-ff75a30a5b71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:29.110251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666b3f8-7d85-4c77-989d-ff75a30a5b71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:29.111470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666b3f8-7d85-4c77-989d-ff75a30a5b71 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:29.122466 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1180/2351] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:52:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:52:29.426299 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee017100-52bb-4da5-a8a4-c089af404c4f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:29.428149 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ee017100-52bb-4da5-a8a4-c089af404c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:29.440887 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee017100-52bb-4da5-a8a4-c089af404c4f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:29.447101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d6104-48a1-43b0-97ea-1ddc369af6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:29.447816 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d6104-48a1-43b0-97ea-1ddc369af6f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:29.448095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d6104-48a1-43b0-97ea-1ddc369af6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:29.448365 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d6104-48a1-43b0-97ea-1ddc369af6f1 admin admin] PATH_INFO: `/v3/projects/15d8f644bc9342379a8c6007f00aa363` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:29.449824 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d6104-48a1-43b0-97ea-1ddc369af6f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15d8f644bc9342379a8c6007f00aa363)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:29.451781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d6104-48a1-43b0-97ea-1ddc369af6f1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:29.488462 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:29.507279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-503d6104-48a1-43b0-97ea-1ddc369af6f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15d8f644bc9342379a8c6007f00aa363', 'request_id': 'req-503d6104-48a1-43b0-97ea-1ddc369af6f1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:29.513966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-503d6104-48a1-43b0-97ea-1ddc369af6f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15d8f644bc9342379a8c6007f00aa363', 'request_id': 'req-503d6104-48a1-43b0-97ea-1ddc369af6f1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:29.523460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-503d6104-48a1-43b0-97ea-1ddc369af6f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15d8f644bc9342379a8c6007f00aa363', 'request_id': 'req-503d6104-48a1-43b0-97ea-1ddc369af6f1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:29.537674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-503d6104-48a1-43b0-97ea-1ddc369af6f1 admin admin] The token cache is being invalidate because project 15d8f644bc9342379a8c6007f00aa363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:52:29.538045 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-503d6104-48a1-43b0-97ea-1ddc369af6f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:29.538697 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-503d6104-48a1-43b0-97ea-1ddc369af6f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:29.540533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1172/2352] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:52:29 2022] DELETE /identity/v3/projects/15d8f644bc9342379a8c6007f00aa363 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:29.546700 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2bc3d-262b-468a-8af2-09cae509145e None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:29.547164 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2bc3d-262b-468a-8af2-09cae509145e None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:29.547511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2bc3d-262b-468a-8af2-09cae509145e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:29.569600 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-f4c2bc3d-262b-468a-8af2-09cae509145e None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:29.571201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f4c2bc3d-262b-468a-8af2-09cae509145e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:29.586372 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:29.658808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1181/2353] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:29.661018 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d00a8-1975-4451-9127-9f9c9da17a1a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:29.661220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d00a8-1975-4451-9127-9f9c9da17a1a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:29.661459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d00a8-1975-4451-9127-9f9c9da17a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:29.686810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-906d00a8-1975-4451-9127-9f9c9da17a1a None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:29.688958 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-906d00a8-1975-4451-9127-9f9c9da17a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:29.711499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1173/2354] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:29.719341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2bc3d-262b-468a-8af2-09cae509145e None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:29.730208 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2bc3d-262b-468a-8af2-09cae509145e None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:29.735124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d02923-677e-4701-afb1-9f8c34a9cf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:29.735743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d02923-677e-4701-afb1-9f8c34a9cf1b admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:29.736048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d02923-677e-4701-afb1-9f8c34a9cf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:29.736321 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d02923-677e-4701-afb1-9f8c34a9cf1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:29.736972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d02923-677e-4701-afb1-9f8c34a9cf1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:52:29.737306 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d02923-677e-4701-afb1-9f8c34a9cf1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:29.739074 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d02923-677e-4701-afb1-9f8c34a9cf1b admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:29.753927 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1182/2355] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:52:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:52:29.756824 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d00a8-1975-4451-9127-9f9c9da17a1a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:29.767190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d00a8-1975-4451-9127-9f9c9da17a1a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:29.772704 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f3a86-7c0b-4b98-b047-6641800eaf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:29.773360 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f3a86-7c0b-4b98-b047-6641800eaf29 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:29.773640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f3a86-7c0b-4b98-b047-6641800eaf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:29.773911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f3a86-7c0b-4b98-b047-6641800eaf29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:29.774561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f3a86-7c0b-4b98-b047-6641800eaf29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:29.776519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f3a86-7c0b-4b98-b047-6641800eaf29 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:29.811005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1174/2356] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:52:29 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:52:29.814894 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d02923-677e-4701-afb1-9f8c34a9cf1b admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:29.825110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d02923-677e-4701-afb1-9f8c34a9cf1b admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:29.888784 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d593699-9133-466b-979b-3b5c0727ffaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:29.889433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d593699-9133-466b-979b-3b5c0727ffaf admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:29.889713 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d593699-9133-466b-979b-3b5c0727ffaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:29.890014 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d593699-9133-466b-979b-3b5c0727ffaf admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:29.890662 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d593699-9133-466b-979b-3b5c0727ffaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:29.892579 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d593699-9133-466b-979b-3b5c0727ffaf admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:29.920928 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1183/2357] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:52:29 2022] POST /identity/v3/users => generated 302 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:52:29.922102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f3a86-7c0b-4b98-b047-6641800eaf29 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:29.932443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f3a86-7c0b-4b98-b047-6641800eaf29 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:29.937244 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f409b-cdfc-4b70-b26e-7f56d0b52b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:29.937864 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f409b-cdfc-4b70-b26e-7f56d0b52b32 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:29.938147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f409b-cdfc-4b70-b26e-7f56d0b52b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:29.938497 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f409b-cdfc-4b70-b26e-7f56d0b52b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:29.939151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f409b-cdfc-4b70-b26e-7f56d0b52b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:29.940783 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8f409b-cdfc-4b70-b26e-7f56d0b52b32 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:29.954802 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1175/2358] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:52:29.956002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d593699-9133-466b-979b-3b5c0727ffaf admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:29.967285 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d593699-9133-466b-979b-3b5c0727ffaf admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:29.973713 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bff638-3213-4788-95ca-18ac5e680442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:29.974503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bff638-3213-4788-95ca-18ac5e680442 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:29.974785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bff638-3213-4788-95ca-18ac5e680442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:29.975052 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bff638-3213-4788-95ca-18ac5e680442 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:29.975702 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bff638-3213-4788-95ca-18ac5e680442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:29.977438 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bff638-3213-4788-95ca-18ac5e680442 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:29.990877 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1184/2359] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:52:29.995018 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f409b-cdfc-4b70-b26e-7f56d0b52b32 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:30.007447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8f409b-cdfc-4b70-b26e-7f56d0b52b32 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:30.012606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3433fa-bf02-4f23-8274-18d5ada108fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:30.013419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3433fa-bf02-4f23-8274-18d5ada108fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:30.013697 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3433fa-bf02-4f23-8274-18d5ada108fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:30.013963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3433fa-bf02-4f23-8274-18d5ada108fe admin admin] PATH_INFO: `/v3/projects/77752fa5f55743a2a34c0227b3d1139f/users/f997bd04eb8d476f8611f040cae29e4c/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:30.031885 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3433fa-bf02-4f23-8274-18d5ada108fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=77752fa5f55743a2a34c0227b3d1139f, user_id=f997bd04eb8d476f8611f040cae29e4c, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:30.034605 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3433fa-bf02-4f23-8274-18d5ada108fe admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:30.050648 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1176/2360] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:52:29 2022] PUT /identity/v3/projects/77752fa5f55743a2a34c0227b3d1139f/users/f997bd04eb8d476f8611f040cae29e4c/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:52:30.064100 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd295248-5081-4e8f-a368-8b1a150dbfcd None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:30.064100 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd295248-5081-4e8f-a368-8b1a150dbfcd None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:30.064816 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd295248-5081-4e8f-a368-8b1a150dbfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:30.097626 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-cd295248-5081-4e8f-a368-8b1a150dbfcd None None] MFA Rules not processed for user `f997bd04eb8d476f8611f040cae29e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:30.099159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-cd295248-5081-4e8f-a368-8b1a150dbfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:30.171629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1185/2361] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:30 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:30.176534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3433fa-bf02-4f23-8274-18d5ada108fe admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:30.259625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3433fa-bf02-4f23-8274-18d5ada108fe admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:30.265361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b6668-67b1-4621-afef-608ef51aaabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:30.266100 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b6668-67b1-4621-afef-608ef51aaabe admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:30.266486 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b6668-67b1-4621-afef-608ef51aaabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:30.266860 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b6668-67b1-4621-afef-608ef51aaabe admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:30.267623 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b6668-67b1-4621-afef-608ef51aaabe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:30.269778 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b6668-67b1-4621-afef-608ef51aaabe admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:30.308522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1177/2362] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:52:30 2022] POST /identity/v3/projects => generated 405 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:52:30.313073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd295248-5081-4e8f-a368-8b1a150dbfcd None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:30.325327 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd295248-5081-4e8f-a368-8b1a150dbfcd None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:30.337248 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:30.398399 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cba007-eecc-4caf-ba52-cc325881d2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:30.399177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cba007-eecc-4caf-ba52-cc325881d2f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:30.399475 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cba007-eecc-4caf-ba52-cc325881d2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:30.399736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cba007-eecc-4caf-ba52-cc325881d2f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:30.400353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cba007-eecc-4caf-ba52-cc325881d2f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:30.402332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cba007-eecc-4caf-ba52-cc325881d2f5 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:30.432733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1186/2363] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:52:30 2022] POST /identity/v3/users => generated 309 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:52:30.434427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b6668-67b1-4621-afef-608ef51aaabe admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:30.446584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b6668-67b1-4621-afef-608ef51aaabe admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:30.452758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c0506-1262-4089-bf2d-79053ad4694f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:30.453426 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c0506-1262-4089-bf2d-79053ad4694f admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:30.453706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c0506-1262-4089-bf2d-79053ad4694f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:30.453983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c0506-1262-4089-bf2d-79053ad4694f admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:30.454648 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c0506-1262-4089-bf2d-79053ad4694f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:30.456302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c0506-1262-4089-bf2d-79053ad4694f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:30.469785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1178/2364] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:52:30.475013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cba007-eecc-4caf-ba52-cc325881d2f5 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:30.492714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cba007-eecc-4caf-ba52-cc325881d2f5 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:30.501191 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b29d8-545b-484f-b03e-4fe362947224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:30.502220 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b29d8-545b-484f-b03e-4fe362947224 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:30.502718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b29d8-545b-484f-b03e-4fe362947224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:30.503222 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b29d8-545b-484f-b03e-4fe362947224 admin admin] PATH_INFO: `/v3/projects/1a10772ff35c408dbd5b9e7a2bcb2c8c/users/7fb5c28a663d44c7a1ceac80b7495e8b/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:30.522049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b29d8-545b-484f-b03e-4fe362947224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a10772ff35c408dbd5b9e7a2bcb2c8c, user_id=7fb5c28a663d44c7a1ceac80b7495e8b, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:30.524953 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b29d8-545b-484f-b03e-4fe362947224 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:30.541412 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1187/2365] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:52:30 2022] PUT /identity/v3/projects/1a10772ff35c408dbd5b9e7a2bcb2c8c/users/7fb5c28a663d44c7a1ceac80b7495e8b/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:30.545751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c0506-1262-4089-bf2d-79053ad4694f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:30.556574 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c0506-1262-4089-bf2d-79053ad4694f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:30.579260 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:30.633187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701a185-35cd-4f36-a367-ed6740535e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:30.633808 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701a185-35cd-4f36-a367-ed6740535e25 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:30.634082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701a185-35cd-4f36-a367-ed6740535e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:30.634348 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701a185-35cd-4f36-a367-ed6740535e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:30.635000 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701a185-35cd-4f36-a367-ed6740535e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:30.636796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701a185-35cd-4f36-a367-ed6740535e25 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:30.650920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1179/2366] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:52:30.654814 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b29d8-545b-484f-b03e-4fe362947224 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:30.667068 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b29d8-545b-484f-b03e-4fe362947224 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:30.674759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ed5b4-772c-4d34-9568-e3eb739dd593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:30.675547 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ed5b4-772c-4d34-9568-e3eb739dd593 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:30.675908 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ed5b4-772c-4d34-9568-e3eb739dd593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:30.676292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ed5b4-772c-4d34-9568-e3eb739dd593 admin admin] PATH_INFO: `/v3/projects/1a10772ff35c408dbd5b9e7a2bcb2c8c/users/7fb5c28a663d44c7a1ceac80b7495e8b/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:30.678441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ed5b4-772c-4d34-9568-e3eb739dd593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a10772ff35c408dbd5b9e7a2bcb2c8c, user_id=7fb5c28a663d44c7a1ceac80b7495e8b, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:30.681334 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ed5b4-772c-4d34-9568-e3eb739dd593 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:30.694127 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1188/2367] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:52:30 2022] PUT /identity/v3/projects/1a10772ff35c408dbd5b9e7a2bcb2c8c/users/7fb5c28a663d44c7a1ceac80b7495e8b/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:52:30.706200 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa2de-60fd-4b71-b3e6-4dc1874a5460 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:30.706435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa2de-60fd-4b71-b3e6-4dc1874a5460 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:30.706746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa2de-60fd-4b71-b3e6-4dc1874a5460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:30.741124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-1c1fa2de-60fd-4b71-b3e6-4dc1874a5460 None None] MFA Rules not processed for user `7fb5c28a663d44c7a1ceac80b7495e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:30.742433 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1c1fa2de-60fd-4b71-b3e6-4dc1874a5460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:30.859477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1180/2368] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:30 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:30.872848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ed5b4-772c-4d34-9568-e3eb739dd593 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:30.874625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4d5ed5b4-772c-4d34-9568-e3eb739dd593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:30.985569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ed5b4-772c-4d34-9568-e3eb739dd593 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:30.992863 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f591f8-9156-438b-83fa-b3d0d978c6aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:30.994349 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f591f8-9156-438b-83fa-b3d0d978c6aa service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:30.995021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f591f8-9156-438b-83fa-b3d0d978c6aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:30.995563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f591f8-9156-438b-83fa-b3d0d978c6aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:31.017728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f591f8-9156-438b-83fa-b3d0d978c6aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:31.019220 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f591f8-9156-438b-83fa-b3d0d978c6aa service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:31.047372 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1189/2369] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:52:30 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:52:31.274433 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fa2de-60fd-4b71-b3e6-4dc1874a5460 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:31.346402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fa2de-60fd-4b71-b3e6-4dc1874a5460 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:31.351721 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e268f1-5949-43a0-9088-ab8aa4b602da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:31.352377 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e268f1-5949-43a0-9088-ab8aa4b602da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:31.352646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e268f1-5949-43a0-9088-ab8aa4b602da admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:31.352933 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e268f1-5949-43a0-9088-ab8aa4b602da admin admin] PATH_INFO: `/v3/users/f997bd04eb8d476f8611f040cae29e4c` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:31.354270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e268f1-5949-43a0-9088-ab8aa4b602da admin admin] RBAC: Authorizing `identity:delete_user(user_id=f997bd04eb8d476f8611f040cae29e4c)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:31.356290 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e268f1-5949-43a0-9088-ab8aa4b602da admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:31.437524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a2e268f1-5949-43a0-9088-ab8aa4b602da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f997bd04eb8d476f8611f040cae29e4c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:31.439287 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:31.450901 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a2e268f1-5949-43a0-9088-ab8aa4b602da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f997bd04eb8d476f8611f040cae29e4c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:31.451624 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a2e268f1-5949-43a0-9088-ab8aa4b602da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f997bd04eb8d476f8611f040cae29e4c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:31.452447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a2e268f1-5949-43a0-9088-ab8aa4b602da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f997bd04eb8d476f8611f040cae29e4c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:31.469868 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a2e268f1-5949-43a0-9088-ab8aa4b602da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f997bd04eb8d476f8611f040cae29e4c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:31.484693 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1181/2370] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:52:31 2022] DELETE /identity/v3/users/f997bd04eb8d476f8611f040cae29e4c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:31.489952 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffae2eb-d5ac-4b94-89e6-3c3ceee2fe75 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:31.490896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffae2eb-d5ac-4b94-89e6-3c3ceee2fe75 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:31.490896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffae2eb-d5ac-4b94-89e6-3c3ceee2fe75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:31.511673 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-3ffae2eb-d5ac-4b94-89e6-3c3ceee2fe75 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:31.512991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-3ffae2eb-d5ac-4b94-89e6-3c3ceee2fe75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:31.565701 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:31.587826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1190/2371] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:31.603930 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e268f1-5949-43a0-9088-ab8aa4b602da admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:31.603930 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a2e268f1-5949-43a0-9088-ab8aa4b602da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:31.673057 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e268f1-5949-43a0-9088-ab8aa4b602da admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:31.677838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebe759-39d7-4f11-8055-41016d0b4d6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:31.678472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebe759-39d7-4f11-8055-41016d0b4d6d service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:31.678747 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebe759-39d7-4f11-8055-41016d0b4d6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:31.679019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebe759-39d7-4f11-8055-41016d0b4d6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:31.690778 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ebe759-39d7-4f11-8055-41016d0b4d6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:31.692583 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ebe759-39d7-4f11-8055-41016d0b4d6d service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:31.705069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1182/2372] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:52:32.026640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffae2eb-d5ac-4b94-89e6-3c3ceee2fe75 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:32.028760 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-3ffae2eb-d5ac-4b94-89e6-3c3ceee2fe75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:32.044541 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffae2eb-d5ac-4b94-89e6-3c3ceee2fe75 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:32.055021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86a31d-6262-4c3c-b28d-d477f07740ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:32.055825 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86a31d-6262-4c3c-b28d-d477f07740ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:32.056408 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86a31d-6262-4c3c-b28d-d477f07740ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:32.056682 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86a31d-6262-4c3c-b28d-d477f07740ed admin admin] PATH_INFO: `/v3/projects/77752fa5f55743a2a34c0227b3d1139f` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:32.058322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86a31d-6262-4c3c-b28d-d477f07740ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=77752fa5f55743a2a34c0227b3d1139f)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:32.060682 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86a31d-6262-4c3c-b28d-d477f07740ed admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:32.132699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1b86a31d-6262-4c3c-b28d-d477f07740ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77752fa5f55743a2a34c0227b3d1139f', 'request_id': 'req-1b86a31d-6262-4c3c-b28d-d477f07740ed'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:32.140425 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1b86a31d-6262-4c3c-b28d-d477f07740ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77752fa5f55743a2a34c0227b3d1139f', 'request_id': 'req-1b86a31d-6262-4c3c-b28d-d477f07740ed'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:32.149087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1b86a31d-6262-4c3c-b28d-d477f07740ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77752fa5f55743a2a34c0227b3d1139f', 'request_id': 'req-1b86a31d-6262-4c3c-b28d-d477f07740ed'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:32.164493 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1b86a31d-6262-4c3c-b28d-d477f07740ed admin admin] The token cache is being invalidate because project 77752fa5f55743a2a34c0227b3d1139f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:52:32.164781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1b86a31d-6262-4c3c-b28d-d477f07740ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:32.165827 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1b86a31d-6262-4c3c-b28d-d477f07740ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:32.167439 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1191/2373] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:52:32 2022] DELETE /identity/v3/projects/77752fa5f55743a2a34c0227b3d1139f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:32.172922 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebe759-39d7-4f11-8055-41016d0b4d6d service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:32.192477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebe759-39d7-4f11-8055-41016d0b4d6d service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:32.268109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c7bea-b32b-4d18-92dd-115b82a7b86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:32.268769 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c7bea-b32b-4d18-92dd-115b82a7b86a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:32.269027 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c7bea-b32b-4d18-92dd-115b82a7b86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:32.269329 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c7bea-b32b-4d18-92dd-115b82a7b86a admin admin] PATH_INFO: `/v3/users/7fb5c28a663d44c7a1ceac80b7495e8b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:32.270512 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c7bea-b32b-4d18-92dd-115b82a7b86a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fb5c28a663d44c7a1ceac80b7495e8b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:32.272480 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c7bea-b32b-4d18-92dd-115b82a7b86a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:32.291639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa43a3-bda6-40a9-a4aa-2263e3eee5fc None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:32.291927 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa43a3-bda6-40a9-a4aa-2263e3eee5fc None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:32.292258 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa43a3-bda6-40a9-a4aa-2263e3eee5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:32.325811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-77aa43a3-bda6-40a9-a4aa-2263e3eee5fc None None] MFA Rules not processed for user `0216bf67389e4627bf953b7dcc97a88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:32.326330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a33c7bea-b32b-4d18-92dd-115b82a7b86a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fb5c28a663d44c7a1ceac80b7495e8b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:32.327040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-77aa43a3-bda6-40a9-a4aa-2263e3eee5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:32.335723 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a33c7bea-b32b-4d18-92dd-115b82a7b86a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fb5c28a663d44c7a1ceac80b7495e8b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:32.336501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a33c7bea-b32b-4d18-92dd-115b82a7b86a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fb5c28a663d44c7a1ceac80b7495e8b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:32.337115 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a33c7bea-b32b-4d18-92dd-115b82a7b86a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fb5c28a663d44c7a1ceac80b7495e8b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:32.349611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:32.351350 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a33c7bea-b32b-4d18-92dd-115b82a7b86a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fb5c28a663d44c7a1ceac80b7495e8b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:32.366304 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1183/2374] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:52:32 2022] DELETE /identity/v3/users/7fb5c28a663d44c7a1ceac80b7495e8b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:32.393212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1192/2375] 10.222.0.57 () {58 vars in 967 bytes} [Mon Sep 26 05:52:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:32.467626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c7bea-b32b-4d18-92dd-115b82a7b86a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:32.469498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a33c7bea-b32b-4d18-92dd-115b82a7b86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:32.572904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c7bea-b32b-4d18-92dd-115b82a7b86a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:32.578942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4e063-b8c7-4877-875b-7b3def07a47e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f258aba84d4dd59ebe8de973ac65c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:32.579575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4e063-b8c7-4877-875b-7b3def07a47e service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:32.579864 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4e063-b8c7-4877-875b-7b3def07a47e service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:32.580136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4e063-b8c7-4877-875b-7b3def07a47e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:32.647307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace4e063-b8c7-4877-875b-7b3def07a47e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:32.649821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace4e063-b8c7-4877-875b-7b3def07a47e service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:32.659848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1184/2376] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 26 05:52:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:52:32.714378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa43a3-bda6-40a9-a4aa-2263e3eee5fc None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:32.717333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-77aa43a3-bda6-40a9-a4aa-2263e3eee5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:32.735719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa43a3-bda6-40a9-a4aa-2263e3eee5fc None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:32.822106 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241aa4f3-5a06-4dfd-9464-d3a2568e3cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:32.822503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241aa4f3-5a06-4dfd-9464-d3a2568e3cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:32.822786 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241aa4f3-5a06-4dfd-9464-d3a2568e3cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:32.823209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241aa4f3-5a06-4dfd-9464-d3a2568e3cd8 admin admin] PATH_INFO: `/v3/projects/1a10772ff35c408dbd5b9e7a2bcb2c8c` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:32.824469 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241aa4f3-5a06-4dfd-9464-d3a2568e3cd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a10772ff35c408dbd5b9e7a2bcb2c8c)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:32.832364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241aa4f3-5a06-4dfd-9464-d3a2568e3cd8 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:32.847677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:32.905895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-241aa4f3-5a06-4dfd-9464-d3a2568e3cd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a10772ff35c408dbd5b9e7a2bcb2c8c', 'request_id': 'req-241aa4f3-5a06-4dfd-9464-d3a2568e3cd8'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:32.917277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-241aa4f3-5a06-4dfd-9464-d3a2568e3cd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a10772ff35c408dbd5b9e7a2bcb2c8c', 'request_id': 'req-241aa4f3-5a06-4dfd-9464-d3a2568e3cd8'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:32.927321 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-241aa4f3-5a06-4dfd-9464-d3a2568e3cd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a10772ff35c408dbd5b9e7a2bcb2c8c', 'request_id': 'req-241aa4f3-5a06-4dfd-9464-d3a2568e3cd8'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:32.940297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-241aa4f3-5a06-4dfd-9464-d3a2568e3cd8 admin admin] The token cache is being invalidate because project 1a10772ff35c408dbd5b9e7a2bcb2c8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:52:32.940589 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-241aa4f3-5a06-4dfd-9464-d3a2568e3cd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:32.941651 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-241aa4f3-5a06-4dfd-9464-d3a2568e3cd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:32.943684 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1193/2377] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:52:32 2022] DELETE /identity/v3/projects/1a10772ff35c408dbd5b9e7a2bcb2c8c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:32.949500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b2112-2b0f-44eb-93af-07b9cc97d990 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:32.949788 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b2112-2b0f-44eb-93af-07b9cc97d990 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:32.950066 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b2112-2b0f-44eb-93af-07b9cc97d990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:32.974812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a39b2112-2b0f-44eb-93af-07b9cc97d990 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:32.975967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a39b2112-2b0f-44eb-93af-07b9cc97d990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:33.041427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1185/2378] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:52:33.049850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdffa6-82e4-4a1a-a325-b6c2ab157bc6 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:33.050217 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdffa6-82e4-4a1a-a325-b6c2ab157bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:33.050374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdffa6-82e4-4a1a-a325-b6c2ab157bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:33.072446 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-7cfdffa6-82e4-4a1a-a325-b6c2ab157bc6 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:33.073711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7cfdffa6-82e4-4a1a-a325-b6c2ab157bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:33.087258 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1194/2379] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:33.092018 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b2112-2b0f-44eb-93af-07b9cc97d990 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:33.107801 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b2112-2b0f-44eb-93af-07b9cc97d990 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:33.115440 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5a725-3edd-4946-a00f-aed8f51e1043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:33.116079 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5a725-3edd-4946-a00f-aed8f51e1043 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:33.116376 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5a725-3edd-4946-a00f-aed8f51e1043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:33.116710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5a725-3edd-4946-a00f-aed8f51e1043 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:33.117861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5a725-3edd-4946-a00f-aed8f51e1043 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:52:33.117861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5a725-3edd-4946-a00f-aed8f51e1043 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:33.119225 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5a725-3edd-4946-a00f-aed8f51e1043 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:33.135124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1186/2380] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:52:33.139724 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdffa6-82e4-4a1a-a325-b6c2ab157bc6 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:33.149365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdffa6-82e4-4a1a-a325-b6c2ab157bc6 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:33.154772 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fb4e1-ed89-469c-94e2-5d75e9c55782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:33.155445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fb4e1-ed89-469c-94e2-5d75e9c55782 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:33.155699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fb4e1-ed89-469c-94e2-5d75e9c55782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:33.155940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3fb4e1-ed89-469c-94e2-5d75e9c55782 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:33.156615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fb4e1-ed89-469c-94e2-5d75e9c55782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:33.158576 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3fb4e1-ed89-469c-94e2-5d75e9c55782 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:33.192527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1195/2381] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:52:33 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:52:33.199982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5a725-3edd-4946-a00f-aed8f51e1043 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:33.213817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5a725-3edd-4946-a00f-aed8f51e1043 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:33.236393 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:33.290921 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb1178-e6bf-4d5c-9624-aaaf17575aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:33.291552 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb1178-e6bf-4d5c-9624-aaaf17575aea admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:33.291822 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb1178-e6bf-4d5c-9624-aaaf17575aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:33.292091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb1178-e6bf-4d5c-9624-aaaf17575aea admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:33.292726 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb1178-e6bf-4d5c-9624-aaaf17575aea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:33.295027 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb1178-e6bf-4d5c-9624-aaaf17575aea admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:33.322446 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1187/2382] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:52:33 2022] POST /identity/v3/users => generated 285 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:52:33.328543 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fb4e1-ed89-469c-94e2-5d75e9c55782 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:33.340996 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3fb4e1-ed89-469c-94e2-5d75e9c55782 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:33.346517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db7167-d4ff-48d0-b501-5b5b840cca38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:33.347566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db7167-d4ff-48d0-b501-5b5b840cca38 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:33.348068 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db7167-d4ff-48d0-b501-5b5b840cca38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:33.348556 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db7167-d4ff-48d0-b501-5b5b840cca38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:33.349877 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db7167-d4ff-48d0-b501-5b5b840cca38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:33.352782 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db7167-d4ff-48d0-b501-5b5b840cca38 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:33.373341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1196/2383] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:52:33.376237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb1178-e6bf-4d5c-9624-aaaf17575aea admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:33.394323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb1178-e6bf-4d5c-9624-aaaf17575aea admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:33.402213 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea5e18-452a-4a8c-9b1a-a305985b3d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:33.403436 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea5e18-452a-4a8c-9b1a-a305985b3d50 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:33.403954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea5e18-452a-4a8c-9b1a-a305985b3d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:33.404449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea5e18-452a-4a8c-9b1a-a305985b3d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:33.405677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea5e18-452a-4a8c-9b1a-a305985b3d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:33.408675 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea5e18-452a-4a8c-9b1a-a305985b3d50 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:33.427329 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1188/2384] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:52:33.434240 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db7167-d4ff-48d0-b501-5b5b840cca38 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:33.447354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db7167-d4ff-48d0-b501-5b5b840cca38 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:33.453590 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c9481-a00f-418c-92d6-d673bba3471a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:33.454571 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c9481-a00f-418c-92d6-d673bba3471a admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:33.454854 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c9481-a00f-418c-92d6-d673bba3471a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:33.455183 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c9481-a00f-418c-92d6-d673bba3471a admin admin] PATH_INFO: `/v3/projects/d482997621c6485cb411541e2b14e2e3/users/acba5ad167964f16b7bf3a33bb99a478/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:33.483396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c9481-a00f-418c-92d6-d673bba3471a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d482997621c6485cb411541e2b14e2e3, user_id=acba5ad167964f16b7bf3a33bb99a478, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:33.488451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c9481-a00f-418c-92d6-d673bba3471a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:33.511249 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1197/2385] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:52:33 2022] PUT /identity/v3/projects/d482997621c6485cb411541e2b14e2e3/users/acba5ad167964f16b7bf3a33bb99a478/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:33.522573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea5103-ea9e-4ab9-8ee8-0655db01b03b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:33.522959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea5103-ea9e-4ab9-8ee8-0655db01b03b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:33.523335 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea5103-ea9e-4ab9-8ee8-0655db01b03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:33.558372 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-97ea5103-ea9e-4ab9-8ee8-0655db01b03b None None] MFA Rules not processed for user `acba5ad167964f16b7bf3a33bb99a478`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:33.559772 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-97ea5103-ea9e-4ab9-8ee8-0655db01b03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:33.633843 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1189/2386] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:33.635440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c9481-a00f-418c-92d6-d673bba3471a admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:33.707152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c9481-a00f-418c-92d6-d673bba3471a admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:33.713817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21574e4c-a4ad-49de-8a54-f5cad2092d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:33.714226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21574e4c-a4ad-49de-8a54-f5cad2092d97 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:33.714412 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21574e4c-a4ad-49de-8a54-f5cad2092d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:33.715377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21574e4c-a4ad-49de-8a54-f5cad2092d97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:33.715377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21574e4c-a4ad-49de-8a54-f5cad2092d97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:33.717322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21574e4c-a4ad-49de-8a54-f5cad2092d97 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:33.751212 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1198/2387] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:52:33 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:52:33.752686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea5103-ea9e-4ab9-8ee8-0655db01b03b None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:33.764766 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea5103-ea9e-4ab9-8ee8-0655db01b03b None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:33.832681 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d816f-3fc7-41c1-8e6f-72a514aff42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:33.833428 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d816f-3fc7-41c1-8e6f-72a514aff42b admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:33.833704 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d816f-3fc7-41c1-8e6f-72a514aff42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:33.834141 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d816f-3fc7-41c1-8e6f-72a514aff42b admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:33.834906 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d816f-3fc7-41c1-8e6f-72a514aff42b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:33.837777 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d816f-3fc7-41c1-8e6f-72a514aff42b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:33.862803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1190/2388] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:52:33 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:52:33.868235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21574e4c-a4ad-49de-8a54-f5cad2092d97 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:33.879548 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21574e4c-a4ad-49de-8a54-f5cad2092d97 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:33.884366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ccaa0-b3d6-4b19-9a53-ba337666b778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:33.885054 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ccaa0-b3d6-4b19-9a53-ba337666b778 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:33.885420 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ccaa0-b3d6-4b19-9a53-ba337666b778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:33.885755 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ccaa0-b3d6-4b19-9a53-ba337666b778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:33.886577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ccaa0-b3d6-4b19-9a53-ba337666b778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:33.888214 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ccaa0-b3d6-4b19-9a53-ba337666b778 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:33.899505 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1199/2389] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:52:33.904300 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d816f-3fc7-41c1-8e6f-72a514aff42b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:33.914642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d816f-3fc7-41c1-8e6f-72a514aff42b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:33.919635 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab697c-458f-43c0-aeb8-85496be95ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:33.920142 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab697c-458f-43c0-aeb8-85496be95ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:33.920417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab697c-458f-43c0-aeb8-85496be95ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:33.920755 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab697c-458f-43c0-aeb8-85496be95ccf admin admin] PATH_INFO: `/v3/projects/136e51ce598d40569ff2e4fb81ace028/users/88f8c014379447c689174454e5cfda0e/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:33.933402 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:33.938966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab697c-458f-43c0-aeb8-85496be95ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=136e51ce598d40569ff2e4fb81ace028, user_id=88f8c014379447c689174454e5cfda0e, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:33.942190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab697c-458f-43c0-aeb8-85496be95ccf admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:33.965653 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1191/2390] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:52:33 2022] PUT /identity/v3/projects/136e51ce598d40569ff2e4fb81ace028/users/88f8c014379447c689174454e5cfda0e/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:52:33.969820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ccaa0-b3d6-4b19-9a53-ba337666b778 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:33.981233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ccaa0-b3d6-4b19-9a53-ba337666b778 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:33.986869 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:34.049172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b9af3-e042-48c0-9afa-a523f67112cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:34.050092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b9af3-e042-48c0-9afa-a523f67112cc admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:34.050373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b9af3-e042-48c0-9afa-a523f67112cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:34.050651 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b9af3-e042-48c0-9afa-a523f67112cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:34.051442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b9af3-e042-48c0-9afa-a523f67112cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:34.052915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b9af3-e042-48c0-9afa-a523f67112cc admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:34.067473 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1200/2391] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:52:34.068832 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab697c-458f-43c0-aeb8-85496be95ccf admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:34.080417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab697c-458f-43c0-aeb8-85496be95ccf admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:34.085687 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03cead-dba6-4a53-bd72-3ea786d472b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:34.086365 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03cead-dba6-4a53-bd72-3ea786d472b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:34.086643 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03cead-dba6-4a53-bd72-3ea786d472b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:34.086917 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03cead-dba6-4a53-bd72-3ea786d472b5 admin admin] PATH_INFO: `/v3/projects/136e51ce598d40569ff2e4fb81ace028/users/88f8c014379447c689174454e5cfda0e/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:34.088860 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03cead-dba6-4a53-bd72-3ea786d472b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=136e51ce598d40569ff2e4fb81ace028, user_id=88f8c014379447c689174454e5cfda0e, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:34.091611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03cead-dba6-4a53-bd72-3ea786d472b5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:34.105525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1192/2392] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:52:34 2022] PUT /identity/v3/projects/136e51ce598d40569ff2e4fb81ace028/users/88f8c014379447c689174454e5cfda0e/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:34.119999 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd11284-0551-4127-8dc8-28137bee575f None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:34.120398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd11284-0551-4127-8dc8-28137bee575f None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:34.120737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd11284-0551-4127-8dc8-28137bee575f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:34.157767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-6dd11284-0551-4127-8dc8-28137bee575f None None] MFA Rules not processed for user `88f8c014379447c689174454e5cfda0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:34.159040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6dd11284-0551-4127-8dc8-28137bee575f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:34.269213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1201/2393] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:34.280747 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03cead-dba6-4a53-bd72-3ea786d472b5 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:34.284038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-df03cead-dba6-4a53-bd72-3ea786d472b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:34.369435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03cead-dba6-4a53-bd72-3ea786d472b5 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:34.373649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e8e13-8091-4eb1-8bb4-2312664c0aec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:34.374473 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e8e13-8091-4eb1-8bb4-2312664c0aec service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:34.374753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e8e13-8091-4eb1-8bb4-2312664c0aec service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:34.375028 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e8e13-8091-4eb1-8bb4-2312664c0aec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:34.386414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e8e13-8091-4eb1-8bb4-2312664c0aec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:34.387938 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e8e13-8091-4eb1-8bb4-2312664c0aec service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:34.398943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1193/2394] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:52:34 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:52:34.455343 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd11284-0551-4127-8dc8-28137bee575f None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:34.457346 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6dd11284-0551-4127-8dc8-28137bee575f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:34.478122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd11284-0551-4127-8dc8-28137bee575f None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:34.483042 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59309311-cf30-4484-99f1-845766603fad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:34.484025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59309311-cf30-4484-99f1-845766603fad service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:34.484315 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59309311-cf30-4484-99f1-845766603fad service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:34.484618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59309311-cf30-4484-99f1-845766603fad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:34.557457 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59309311-cf30-4484-99f1-845766603fad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:34.559322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59309311-cf30-4484-99f1-845766603fad service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:34.570129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1202/2395] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:52:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:52:38.220595 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e8e13-8091-4eb1-8bb4-2312664c0aec service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:38.325524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e8e13-8091-4eb1-8bb4-2312664c0aec service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:38.331996 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767aa8dd-70cd-4c16-87c4-11c3e021ba96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:38.332655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767aa8dd-70cd-4c16-87c4-11c3e021ba96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:38.332934 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767aa8dd-70cd-4c16-87c4-11c3e021ba96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:38.333425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767aa8dd-70cd-4c16-87c4-11c3e021ba96 admin admin] PATH_INFO: `/v3/users/acba5ad167964f16b7bf3a33bb99a478` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:38.334684 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767aa8dd-70cd-4c16-87c4-11c3e021ba96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acba5ad167964f16b7bf3a33bb99a478)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:38.337968 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767aa8dd-70cd-4c16-87c4-11c3e021ba96 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:38.389081 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-767aa8dd-70cd-4c16-87c4-11c3e021ba96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acba5ad167964f16b7bf3a33bb99a478'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:38.397437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-767aa8dd-70cd-4c16-87c4-11c3e021ba96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acba5ad167964f16b7bf3a33bb99a478'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:38.398030 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-767aa8dd-70cd-4c16-87c4-11c3e021ba96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acba5ad167964f16b7bf3a33bb99a478'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:38.398625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-767aa8dd-70cd-4c16-87c4-11c3e021ba96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acba5ad167964f16b7bf3a33bb99a478'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:38.411494 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-767aa8dd-70cd-4c16-87c4-11c3e021ba96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acba5ad167964f16b7bf3a33bb99a478'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:38.420324 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:38.426180 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1194/2396] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:52:38 2022] DELETE /identity/v3/users/acba5ad167964f16b7bf3a33bb99a478 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:38.431302 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d5bb2-521d-4fe0-ae2c-cddfaabc3f3d None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:38.431824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d5bb2-521d-4fe0-ae2c-cddfaabc3f3d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:38.431925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d5bb2-521d-4fe0-ae2c-cddfaabc3f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:38.450795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-681d5bb2-521d-4fe0-ae2c-cddfaabc3f3d None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:38.451993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-681d5bb2-521d-4fe0-ae2c-cddfaabc3f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:38.476689 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:38.513290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1203/2397] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:38.524044 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767aa8dd-70cd-4c16-87c4-11c3e021ba96 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:38.525845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-767aa8dd-70cd-4c16-87c4-11c3e021ba96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:38.591507 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767aa8dd-70cd-4c16-87c4-11c3e021ba96 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:38.595990 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217d06d-d093-4c83-b75a-38d241e01b33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:38.596685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217d06d-d093-4c83-b75a-38d241e01b33 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:38.596818 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217d06d-d093-4c83-b75a-38d241e01b33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:38.597120 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217d06d-d093-4c83-b75a-38d241e01b33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:38.607071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217d06d-d093-4c83-b75a-38d241e01b33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:38.609675 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217d06d-d093-4c83-b75a-38d241e01b33 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:38.621524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1195/2398] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:52:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:52:39.069813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d5bb2-521d-4fe0-ae2c-cddfaabc3f3d None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:39.071277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-681d5bb2-521d-4fe0-ae2c-cddfaabc3f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:39.084497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d5bb2-521d-4fe0-ae2c-cddfaabc3f3d None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:39.089681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066d361-3ead-4b78-aab2-6985d5d2b860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:39.090354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066d361-3ead-4b78-aab2-6985d5d2b860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:39.090627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066d361-3ead-4b78-aab2-6985d5d2b860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:39.090893 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066d361-3ead-4b78-aab2-6985d5d2b860 admin admin] PATH_INFO: `/v3/projects/d482997621c6485cb411541e2b14e2e3` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:39.092013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066d361-3ead-4b78-aab2-6985d5d2b860 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d482997621c6485cb411541e2b14e2e3)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:39.094173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066d361-3ead-4b78-aab2-6985d5d2b860 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:39.145629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d066d361-3ead-4b78-aab2-6985d5d2b860 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd482997621c6485cb411541e2b14e2e3', 'request_id': 'req-d066d361-3ead-4b78-aab2-6985d5d2b860'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:39.152534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d066d361-3ead-4b78-aab2-6985d5d2b860 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd482997621c6485cb411541e2b14e2e3', 'request_id': 'req-d066d361-3ead-4b78-aab2-6985d5d2b860'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:39.162827 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d066d361-3ead-4b78-aab2-6985d5d2b860 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd482997621c6485cb411541e2b14e2e3', 'request_id': 'req-d066d361-3ead-4b78-aab2-6985d5d2b860'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:39.176045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d066d361-3ead-4b78-aab2-6985d5d2b860 admin admin] The token cache is being invalidate because project d482997621c6485cb411541e2b14e2e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:52:39.176477 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d066d361-3ead-4b78-aab2-6985d5d2b860 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:39.178237 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-d066d361-3ead-4b78-aab2-6985d5d2b860 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:39.180647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1204/2399] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:52:39 2022] DELETE /identity/v3/projects/d482997621c6485cb411541e2b14e2e3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:39.184569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217d06d-d093-4c83-b75a-38d241e01b33 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:39.194759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217d06d-d093-4c83-b75a-38d241e01b33 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:39.278473 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170cc47-ed59-4b6a-91b6-83782ceb377f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:39.279228 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170cc47-ed59-4b6a-91b6-83782ceb377f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:39.280416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170cc47-ed59-4b6a-91b6-83782ceb377f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:39.280808 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170cc47-ed59-4b6a-91b6-83782ceb377f admin admin] PATH_INFO: `/v3/users/88f8c014379447c689174454e5cfda0e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:39.282513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170cc47-ed59-4b6a-91b6-83782ceb377f admin admin] RBAC: Authorizing `identity:delete_user(user_id=88f8c014379447c689174454e5cfda0e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:39.285017 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170cc47-ed59-4b6a-91b6-83782ceb377f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:39.351474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9170cc47-ed59-4b6a-91b6-83782ceb377f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88f8c014379447c689174454e5cfda0e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:39.359357 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9170cc47-ed59-4b6a-91b6-83782ceb377f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88f8c014379447c689174454e5cfda0e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:39.360061 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9170cc47-ed59-4b6a-91b6-83782ceb377f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88f8c014379447c689174454e5cfda0e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:39.360675 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9170cc47-ed59-4b6a-91b6-83782ceb377f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88f8c014379447c689174454e5cfda0e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:39.374262 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9170cc47-ed59-4b6a-91b6-83782ceb377f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88f8c014379447c689174454e5cfda0e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:39.387332 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1196/2400] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:52:39 2022] DELETE /identity/v3/users/88f8c014379447c689174454e5cfda0e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:39.705241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066d361-3ead-4b78-aab2-6985d5d2b860 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:39.706757 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d066d361-3ead-4b78-aab2-6985d5d2b860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:39.728163 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066d361-3ead-4b78-aab2-6985d5d2b860 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:39.810783 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f3a28-b1e1-4a8b-afaa-24d2ffae6e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:39.811887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f3a28-b1e1-4a8b-afaa-24d2ffae6e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:39.812725 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f3a28-b1e1-4a8b-afaa-24d2ffae6e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:39.812725 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f3a28-b1e1-4a8b-afaa-24d2ffae6e4d admin admin] PATH_INFO: `/v3/projects/136e51ce598d40569ff2e4fb81ace028` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:39.813893 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f3a28-b1e1-4a8b-afaa-24d2ffae6e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=136e51ce598d40569ff2e4fb81ace028)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:39.815987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f3a28-b1e1-4a8b-afaa-24d2ffae6e4d admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:39.887750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-464f3a28-b1e1-4a8b-afaa-24d2ffae6e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '136e51ce598d40569ff2e4fb81ace028', 'request_id': 'req-464f3a28-b1e1-4a8b-afaa-24d2ffae6e4d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:39.896108 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-464f3a28-b1e1-4a8b-afaa-24d2ffae6e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '136e51ce598d40569ff2e4fb81ace028', 'request_id': 'req-464f3a28-b1e1-4a8b-afaa-24d2ffae6e4d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:39.898179 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:39.909566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-464f3a28-b1e1-4a8b-afaa-24d2ffae6e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '136e51ce598d40569ff2e4fb81ace028', 'request_id': 'req-464f3a28-b1e1-4a8b-afaa-24d2ffae6e4d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:39.928188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-464f3a28-b1e1-4a8b-afaa-24d2ffae6e4d admin admin] The token cache is being invalidate because project 136e51ce598d40569ff2e4fb81ace028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:52:39.928510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-464f3a28-b1e1-4a8b-afaa-24d2ffae6e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:39.929274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-464f3a28-b1e1-4a8b-afaa-24d2ffae6e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:39.931080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1205/2401] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:52:39 2022] DELETE /identity/v3/projects/136e51ce598d40569ff2e4fb81ace028 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:39.936610 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9761c-94c9-4ed8-9d28-5bf52036f2e9 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:39.936965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9761c-94c9-4ed8-9d28-5bf52036f2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:39.937344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9761c-94c9-4ed8-9d28-5bf52036f2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:39.964109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-38d9761c-94c9-4ed8-9d28-5bf52036f2e9 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:39.965479 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-38d9761c-94c9-4ed8-9d28-5bf52036f2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:39.983705 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:40.060613 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1197/2402] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:40.063204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86486732-d480-4e6b-85ef-fdcfa79bba5c None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:40.064714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86486732-d480-4e6b-85ef-fdcfa79bba5c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:40.065021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86486732-d480-4e6b-85ef-fdcfa79bba5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:40.093662 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-86486732-d480-4e6b-85ef-fdcfa79bba5c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:40.096116 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-86486732-d480-4e6b-85ef-fdcfa79bba5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:40.111231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1206/2403] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:40.112026 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9761c-94c9-4ed8-9d28-5bf52036f2e9 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:40.124094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9761c-94c9-4ed8-9d28-5bf52036f2e9 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:40.128888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff6a74-b017-4fe5-955a-1ab50f55cdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:40.129542 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff6a74-b017-4fe5-955a-1ab50f55cdbc admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:40.129819 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff6a74-b017-4fe5-955a-1ab50f55cdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:40.130087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff6a74-b017-4fe5-955a-1ab50f55cdbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:40.130738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff6a74-b017-4fe5-955a-1ab50f55cdbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:52:40.131039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff6a74-b017-4fe5-955a-1ab50f55cdbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:40.132640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff6a74-b017-4fe5-955a-1ab50f55cdbc admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:40.149013 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1198/2404] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:52:40.154152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86486732-d480-4e6b-85ef-fdcfa79bba5c None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:40.168461 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86486732-d480-4e6b-85ef-fdcfa79bba5c None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:40.183313 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a36b2-92ef-4a09-a0e6-f5c8554b8eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:40.184331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a36b2-92ef-4a09-a0e6-f5c8554b8eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:40.184906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a36b2-92ef-4a09-a0e6-f5c8554b8eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:40.185723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a36b2-92ef-4a09-a0e6-f5c8554b8eb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:40.186410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a36b2-92ef-4a09-a0e6-f5c8554b8eb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:40.189462 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a36b2-92ef-4a09-a0e6-f5c8554b8eb8 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:40.251000 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1207/2405] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:52:40 2022] POST /identity/v3/projects => generated 391 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:52:40.252115 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff6a74-b017-4fe5-955a-1ab50f55cdbc admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:40.266249 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff6a74-b017-4fe5-955a-1ab50f55cdbc admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:40.350723 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2931f8c-edd5-48aa-8cb8-c092a8fe78de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:40.351365 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2931f8c-edd5-48aa-8cb8-c092a8fe78de admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:40.351598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2931f8c-edd5-48aa-8cb8-c092a8fe78de admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:40.352126 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2931f8c-edd5-48aa-8cb8-c092a8fe78de admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:40.352510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2931f8c-edd5-48aa-8cb8-c092a8fe78de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:40.354463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2931f8c-edd5-48aa-8cb8-c092a8fe78de admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:40.378189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1199/2406] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:52:40 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:52:40.382752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a36b2-92ef-4a09-a0e6-f5c8554b8eb8 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:40.394526 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a36b2-92ef-4a09-a0e6-f5c8554b8eb8 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:40.400147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13884e-3b08-45a9-963c-ff42be0c1741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:40.400774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13884e-3b08-45a9-963c-ff42be0c1741 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:40.401051 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13884e-3b08-45a9-963c-ff42be0c1741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:40.401367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13884e-3b08-45a9-963c-ff42be0c1741 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:40.402034 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13884e-3b08-45a9-963c-ff42be0c1741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:40.403718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13884e-3b08-45a9-963c-ff42be0c1741 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:40.414994 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1208/2407] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:52:40.419218 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2931f8c-edd5-48aa-8cb8-c092a8fe78de admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:40.430085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2931f8c-edd5-48aa-8cb8-c092a8fe78de admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:40.434918 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90962dc3-a8c7-46c8-ae21-4e6ea851afce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:40.435521 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90962dc3-a8c7-46c8-ae21-4e6ea851afce admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:40.435802 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90962dc3-a8c7-46c8-ae21-4e6ea851afce admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:40.436069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90962dc3-a8c7-46c8-ae21-4e6ea851afce admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:40.436706 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90962dc3-a8c7-46c8-ae21-4e6ea851afce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:40.438370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90962dc3-a8c7-46c8-ae21-4e6ea851afce admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:40.449522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1200/2408] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:52:40.453807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13884e-3b08-45a9-963c-ff42be0c1741 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:40.465003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13884e-3b08-45a9-963c-ff42be0c1741 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:40.470553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d61d21-4c00-4b59-b0a0-aef7ed41e1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:40.471299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d61d21-4c00-4b59-b0a0-aef7ed41e1a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:40.471576 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d61d21-4c00-4b59-b0a0-aef7ed41e1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:40.471849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d61d21-4c00-4b59-b0a0-aef7ed41e1a4 admin admin] PATH_INFO: `/v3/projects/b659b79132d64a5aa06c9e6bd130f79c/users/53ea2d88389c424aaf6cfed76aafc39d/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:40.484187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90962dc3-a8c7-46c8-ae21-4e6ea851afce admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:40.486233 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-90962dc3-a8c7-46c8-ae21-4e6ea851afce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:40.494509 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d61d21-4c00-4b59-b0a0-aef7ed41e1a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b659b79132d64a5aa06c9e6bd130f79c, user_id=53ea2d88389c424aaf6cfed76aafc39d, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:40.501200 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90962dc3-a8c7-46c8-ae21-4e6ea851afce admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:40.501561 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d61d21-4c00-4b59-b0a0-aef7ed41e1a4 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:40.507786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ece6a3-4b43-475e-a644-5d5a44a4fd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:40.508427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ece6a3-4b43-475e-a644-5d5a44a4fd82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:40.508701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ece6a3-4b43-475e-a644-5d5a44a4fd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:40.509021 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ece6a3-4b43-475e-a644-5d5a44a4fd82 admin admin] PATH_INFO: `/v3/users/3cab2568112f46bca576f17916887aa7` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:40.510314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ece6a3-4b43-475e-a644-5d5a44a4fd82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cab2568112f46bca576f17916887aa7)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:40.512375 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ece6a3-4b43-475e-a644-5d5a44a4fd82 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:40.521036 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1209/2409] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:52:40 2022] PUT /identity/v3/projects/b659b79132d64a5aa06c9e6bd130f79c/users/53ea2d88389c424aaf6cfed76aafc39d/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:40.532752 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28661afb-a41c-4289-a6b3-ce7ccba3319b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:40.534263 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28661afb-a41c-4289-a6b3-ce7ccba3319b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:40.534637 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28661afb-a41c-4289-a6b3-ce7ccba3319b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:40.566654 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f6ece6a3-4b43-475e-a644-5d5a44a4fd82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cab2568112f46bca576f17916887aa7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:40.572702 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-28661afb-a41c-4289-a6b3-ce7ccba3319b None None] MFA Rules not processed for user `53ea2d88389c424aaf6cfed76aafc39d`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:40.574035 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-28661afb-a41c-4289-a6b3-ce7ccba3319b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:40.574763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f6ece6a3-4b43-475e-a644-5d5a44a4fd82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cab2568112f46bca576f17916887aa7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:40.575411 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f6ece6a3-4b43-475e-a644-5d5a44a4fd82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cab2568112f46bca576f17916887aa7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:40.576067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f6ece6a3-4b43-475e-a644-5d5a44a4fd82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cab2568112f46bca576f17916887aa7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:40.592879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-f6ece6a3-4b43-475e-a644-5d5a44a4fd82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cab2568112f46bca576f17916887aa7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:40.602003 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:40.607664 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1201/2410] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:52:40 2022] DELETE /identity/v3/users/3cab2568112f46bca576f17916887aa7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:40.649923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1210/2411] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:52:40.656641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ece6a3-4b43-475e-a644-5d5a44a4fd82 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:40.656641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f6ece6a3-4b43-475e-a644-5d5a44a4fd82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:40.735282 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ece6a3-4b43-475e-a644-5d5a44a4fd82 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:40.740421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd82b9d-af07-43bb-9d42-15b80ea41109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:40.741040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd82b9d-af07-43bb-9d42-15b80ea41109 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:40.741354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd82b9d-af07-43bb-9d42-15b80ea41109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:40.741628 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd82b9d-af07-43bb-9d42-15b80ea41109 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:40.742266 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd82b9d-af07-43bb-9d42-15b80ea41109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:40.744196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd82b9d-af07-43bb-9d42-15b80ea41109 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:40.777177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1202/2412] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:52:40 2022] POST /identity/v3/projects => generated 393 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:52:40.780591 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28661afb-a41c-4289-a6b3-ce7ccba3319b None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:40.823864 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd82b9d-af07-43bb-9d42-15b80ea41109 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:40.828538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cfd82b9d-af07-43bb-9d42-15b80ea41109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:40.857549 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28661afb-a41c-4289-a6b3-ce7ccba3319b None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:40.870727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:40.900880 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd82b9d-af07-43bb-9d42-15b80ea41109 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:40.905817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ab8e4-e654-4665-95e2-f3a83f60a31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:40.906459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ab8e4-e654-4665-95e2-f3a83f60a31a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:40.906736 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ab8e4-e654-4665-95e2-f3a83f60a31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:40.907052 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ab8e4-e654-4665-95e2-f3a83f60a31a admin admin] PATH_INFO: `/v3/projects/0b095f1ee00e46cfabebc2dc34992997` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:40.908211 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ab8e4-e654-4665-95e2-f3a83f60a31a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b095f1ee00e46cfabebc2dc34992997)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:40.910211 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ab8e4-e654-4665-95e2-f3a83f60a31a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:40.926044 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b8415-f7df-4b52-8620-e71dc6052951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:40.926689 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b8415-f7df-4b52-8620-e71dc6052951 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:40.926951 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b8415-f7df-4b52-8620-e71dc6052951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:40.927222 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b8415-f7df-4b52-8620-e71dc6052951 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:40.927855 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b8415-f7df-4b52-8620-e71dc6052951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:40.929836 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b8415-f7df-4b52-8620-e71dc6052951 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:40.966748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1211/2413] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:52:40 2022] POST /identity/v3/users => generated 303 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:52:40.971507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b8415-f7df-4b52-8620-e71dc6052951 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:40.986481 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-617ab8e4-e654-4665-95e2-f3a83f60a31a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b095f1ee00e46cfabebc2dc34992997', 'request_id': 'req-617ab8e4-e654-4665-95e2-f3a83f60a31a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:40.991990 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b8415-f7df-4b52-8620-e71dc6052951 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:40.994968 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-617ab8e4-e654-4665-95e2-f3a83f60a31a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b095f1ee00e46cfabebc2dc34992997', 'request_id': 'req-617ab8e4-e654-4665-95e2-f3a83f60a31a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:41.003919 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-617ab8e4-e654-4665-95e2-f3a83f60a31a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b095f1ee00e46cfabebc2dc34992997', 'request_id': 'req-617ab8e4-e654-4665-95e2-f3a83f60a31a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:41.017434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-617ab8e4-e654-4665-95e2-f3a83f60a31a admin admin] The token cache is being invalidate because project 0b095f1ee00e46cfabebc2dc34992997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:52:41.018000 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-617ab8e4-e654-4665-95e2-f3a83f60a31a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:41.018759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-617ab8e4-e654-4665-95e2-f3a83f60a31a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:41.020409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1203/2414] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:52:40 2022] DELETE /identity/v3/projects/0b095f1ee00e46cfabebc2dc34992997 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:41.025003 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ab8e4-e654-4665-95e2-f3a83f60a31a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:41.026761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-617ab8e4-e654-4665-95e2-f3a83f60a31a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:41.044273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:41.064507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ba28b-0b5b-4114-b79e-26d0d9c0659b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:41.065201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ba28b-0b5b-4114-b79e-26d0d9c0659b admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:41.065657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ba28b-0b5b-4114-b79e-26d0d9c0659b admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:41.066085 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ba28b-0b5b-4114-b79e-26d0d9c0659b admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:41.066839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ba28b-0b5b-4114-b79e-26d0d9c0659b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:41.068548 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ba28b-0b5b-4114-b79e-26d0d9c0659b admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:41.079113 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1212/2415] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:52:41.083421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ba28b-0b5b-4114-b79e-26d0d9c0659b admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:41.085481 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f10ba28b-0b5b-4114-b79e-26d0d9c0659b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:41.098494 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ba28b-0b5b-4114-b79e-26d0d9c0659b admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:41.103222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ab8e4-e654-4665-95e2-f3a83f60a31a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:41.103475 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897721a7-573f-4fbd-ae9c-adee66a3afad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:41.104155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897721a7-573f-4fbd-ae9c-adee66a3afad admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:41.104443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897721a7-573f-4fbd-ae9c-adee66a3afad admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:41.104705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897721a7-573f-4fbd-ae9c-adee66a3afad admin admin] PATH_INFO: `/v3/projects/5e18e0d1e77141729181e1f520adeedc/users/a1477ed537e042bc99da2b2ab89a3c5f/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:41.108220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99202674-a744-4ad7-945b-803b41c82c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:41.108868 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99202674-a744-4ad7-945b-803b41c82c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:41.109172 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99202674-a744-4ad7-945b-803b41c82c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:41.109451 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99202674-a744-4ad7-945b-803b41c82c3e admin admin] PATH_INFO: `/v3/users/9b88e498fb9049af99d3c802af19bb73` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:41.110616 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99202674-a744-4ad7-945b-803b41c82c3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b88e498fb9049af99d3c802af19bb73)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:41.112581 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99202674-a744-4ad7-945b-803b41c82c3e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:41.124614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897721a7-573f-4fbd-ae9c-adee66a3afad admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e18e0d1e77141729181e1f520adeedc, user_id=a1477ed537e042bc99da2b2ab89a3c5f, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:41.127426 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897721a7-573f-4fbd-ae9c-adee66a3afad admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:41.142272 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1213/2416] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:52:41 2022] PUT /identity/v3/projects/5e18e0d1e77141729181e1f520adeedc/users/a1477ed537e042bc99da2b2ab89a3c5f/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:52:41.147627 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897721a7-573f-4fbd-ae9c-adee66a3afad admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:41.166033 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-99202674-a744-4ad7-945b-803b41c82c3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b88e498fb9049af99d3c802af19bb73'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:41.174313 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-99202674-a744-4ad7-945b-803b41c82c3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b88e498fb9049af99d3c802af19bb73'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:41.175102 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-99202674-a744-4ad7-945b-803b41c82c3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b88e498fb9049af99d3c802af19bb73'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:41.175717 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-99202674-a744-4ad7-945b-803b41c82c3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b88e498fb9049af99d3c802af19bb73'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:41.189890 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-99202674-a744-4ad7-945b-803b41c82c3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b88e498fb9049af99d3c802af19bb73'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:41.202681 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1204/2417] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:52:41 2022] DELETE /identity/v3/users/9b88e498fb9049af99d3c802af19bb73 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:41.219915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897721a7-573f-4fbd-ae9c-adee66a3afad admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:41.242493 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:41.287144 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff48e9-4e35-4d86-92d6-9edf6f8db288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:41.287762 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff48e9-4e35-4d86-92d6-9edf6f8db288 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:41.288102 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff48e9-4e35-4d86-92d6-9edf6f8db288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:41.288394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff48e9-4e35-4d86-92d6-9edf6f8db288 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:41.289337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff48e9-4e35-4d86-92d6-9edf6f8db288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:41.290934 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff48e9-4e35-4d86-92d6-9edf6f8db288 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:41.301785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1214/2418] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:52:41.306213 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99202674-a744-4ad7-945b-803b41c82c3e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:41.307703 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-99202674-a744-4ad7-945b-803b41c82c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:41.322349 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99202674-a744-4ad7-945b-803b41c82c3e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:41.327935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e060b2c-99f0-4b3d-a955-72ddd99fed8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:41.328563 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e060b2c-99f0-4b3d-a955-72ddd99fed8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:41.328970 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e060b2c-99f0-4b3d-a955-72ddd99fed8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:41.329093 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e060b2c-99f0-4b3d-a955-72ddd99fed8e admin admin] PATH_INFO: `/v3/projects/5e18e0d1e77141729181e1f520adeedc/users/a1477ed537e042bc99da2b2ab89a3c5f/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:41.331180 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e060b2c-99f0-4b3d-a955-72ddd99fed8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e18e0d1e77141729181e1f520adeedc, user_id=a1477ed537e042bc99da2b2ab89a3c5f, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:41.333860 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e060b2c-99f0-4b3d-a955-72ddd99fed8e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:41.345920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1205/2419] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:52:41 2022] PUT /identity/v3/projects/5e18e0d1e77141729181e1f520adeedc/users/a1477ed537e042bc99da2b2ab89a3c5f/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:41.356700 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc64fd-e1d9-4992-9b25-33b5820fe38d None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:41.357177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc64fd-e1d9-4992-9b25-33b5820fe38d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:41.357460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc64fd-e1d9-4992-9b25-33b5820fe38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:41.392806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-f1fc64fd-e1d9-4992-9b25-33b5820fe38d None None] MFA Rules not processed for user `a1477ed537e042bc99da2b2ab89a3c5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:41.394181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f1fc64fd-e1d9-4992-9b25-33b5820fe38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:41.401062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e060b2c-99f0-4b3d-a955-72ddd99fed8e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:41.403565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7e060b2c-99f0-4b3d-a955-72ddd99fed8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:41.470837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1215/2420] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:41 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:41.481401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e060b2c-99f0-4b3d-a955-72ddd99fed8e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:41.482092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc64fd-e1d9-4992-9b25-33b5820fe38d None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:41.483465 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f1fc64fd-e1d9-4992-9b25-33b5820fe38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:41.486129 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93adb7e1-85bc-4857-a8a3-27cebe8b4145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:41.486769 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adb7e1-85bc-4857-a8a3-27cebe8b4145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:41.487050 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adb7e1-85bc-4857-a8a3-27cebe8b4145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:41.487322 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adb7e1-85bc-4857-a8a3-27cebe8b4145 admin admin] PATH_INFO: `/v3/projects/d7219cb9801c4589b04ed212e3e06e08` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:41.488500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93adb7e1-85bc-4857-a8a3-27cebe8b4145 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7219cb9801c4589b04ed212e3e06e08)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:41.490711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93adb7e1-85bc-4857-a8a3-27cebe8b4145 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:41.515622 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:41.546450 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-93adb7e1-85bc-4857-a8a3-27cebe8b4145 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7219cb9801c4589b04ed212e3e06e08', 'request_id': 'req-93adb7e1-85bc-4857-a8a3-27cebe8b4145'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:41.547788 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc64fd-e1d9-4992-9b25-33b5820fe38d None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:41.553623 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-93adb7e1-85bc-4857-a8a3-27cebe8b4145 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7219cb9801c4589b04ed212e3e06e08', 'request_id': 'req-93adb7e1-85bc-4857-a8a3-27cebe8b4145'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:41.562641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-93adb7e1-85bc-4857-a8a3-27cebe8b4145 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7219cb9801c4589b04ed212e3e06e08', 'request_id': 'req-93adb7e1-85bc-4857-a8a3-27cebe8b4145'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:41.573445 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-93adb7e1-85bc-4857-a8a3-27cebe8b4145 admin admin] The token cache is being invalidate because project d7219cb9801c4589b04ed212e3e06e08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:52:41.573730 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-93adb7e1-85bc-4857-a8a3-27cebe8b4145 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:41.574529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-93adb7e1-85bc-4857-a8a3-27cebe8b4145 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:41.576423 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1206/2421] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:52:41 2022] DELETE /identity/v3/projects/d7219cb9801c4589b04ed212e3e06e08 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:52:41.602088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9704b2-84d6-498d-97d8-41f82e2aeb39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:41.602746 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9704b2-84d6-498d-97d8-41f82e2aeb39 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:41.603012 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9704b2-84d6-498d-97d8-41f82e2aeb39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:41.603285 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9704b2-84d6-498d-97d8-41f82e2aeb39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:41.606259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7a9704b2-84d6-498d-97d8-41f82e2aeb39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:41.623875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:41.682445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9704b2-84d6-498d-97d8-41f82e2aeb39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:41.684102 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9704b2-84d6-498d-97d8-41f82e2aeb39 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:41.698168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1216/2422] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:52:41 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:52:41.747923 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93adb7e1-85bc-4857-a8a3-27cebe8b4145 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:41.749513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-93adb7e1-85bc-4857-a8a3-27cebe8b4145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:41.763472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93adb7e1-85bc-4857-a8a3-27cebe8b4145 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:41.767990 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb377e-d7be-44cb-b406-6e68eb103b89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:41.768617 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb377e-d7be-44cb-b406-6e68eb103b89 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:41.768891 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb377e-d7be-44cb-b406-6e68eb103b89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:41.769183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb377e-d7be-44cb-b406-6e68eb103b89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:41.771122 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0aeb377e-d7be-44cb-b406-6e68eb103b89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:41.837782 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb377e-d7be-44cb-b406-6e68eb103b89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:41.839201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb377e-d7be-44cb-b406-6e68eb103b89 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:41.849597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1207/2423] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:52:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:52:45.304213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9704b2-84d6-498d-97d8-41f82e2aeb39 service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:45.305706 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7a9704b2-84d6-498d-97d8-41f82e2aeb39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:45.383371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9704b2-84d6-498d-97d8-41f82e2aeb39 service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:45.388861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca3e22-cc8f-43fd-bd21-46fa2fc991ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:45.389572 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca3e22-cc8f-43fd-bd21-46fa2fc991ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:45.389850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca3e22-cc8f-43fd-bd21-46fa2fc991ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:45.390125 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca3e22-cc8f-43fd-bd21-46fa2fc991ab admin admin] PATH_INFO: `/v3/users/53ea2d88389c424aaf6cfed76aafc39d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:45.391345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca3e22-cc8f-43fd-bd21-46fa2fc991ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=53ea2d88389c424aaf6cfed76aafc39d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:45.393385 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca3e22-cc8f-43fd-bd21-46fa2fc991ab admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:45.449380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-32ca3e22-cc8f-43fd-bd21-46fa2fc991ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53ea2d88389c424aaf6cfed76aafc39d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:45.459426 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-32ca3e22-cc8f-43fd-bd21-46fa2fc991ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53ea2d88389c424aaf6cfed76aafc39d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:45.460529 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-32ca3e22-cc8f-43fd-bd21-46fa2fc991ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53ea2d88389c424aaf6cfed76aafc39d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:45.461245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-32ca3e22-cc8f-43fd-bd21-46fa2fc991ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53ea2d88389c424aaf6cfed76aafc39d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:45.470082 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-32ca3e22-cc8f-43fd-bd21-46fa2fc991ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53ea2d88389c424aaf6cfed76aafc39d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:45.491499 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1217/2424] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:52:45 2022] DELETE /identity/v3/users/53ea2d88389c424aaf6cfed76aafc39d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:45.496892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b2b2b-a7aa-4878-acf8-1865740efbd1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:45.497406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b2b2b-a7aa-4878-acf8-1865740efbd1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:45.497501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b2b2b-a7aa-4878-acf8-1865740efbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:45.518389 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a41b2b2b-a7aa-4878-acf8-1865740efbd1 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:45.519655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a41b2b2b-a7aa-4878-acf8-1865740efbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:45.584082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1208/2425] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:45.594568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca3e22-cc8f-43fd-bd21-46fa2fc991ab admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:45.596172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-32ca3e22-cc8f-43fd-bd21-46fa2fc991ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:45.632386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:45.658667 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca3e22-cc8f-43fd-bd21-46fa2fc991ab admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:45.662520 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ad0cc-6d65-4106-9dec-1ccf6bc200aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:45.663133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ad0cc-6d65-4106-9dec-1ccf6bc200aa service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:45.663407 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ad0cc-6d65-4106-9dec-1ccf6bc200aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:45.663688 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ad0cc-6d65-4106-9dec-1ccf6bc200aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:45.673497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ad0cc-6d65-4106-9dec-1ccf6bc200aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:45.674943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ad0cc-6d65-4106-9dec-1ccf6bc200aa service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:45.685160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1218/2426] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:52:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:52:45.988773 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b2b2b-a7aa-4878-acf8-1865740efbd1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:45.990351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a41b2b2b-a7aa-4878-acf8-1865740efbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:46.003183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b2b2b-a7aa-4878-acf8-1865740efbd1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:46.010073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:46.010849 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:46.011132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:46.011451 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051 admin admin] PATH_INFO: `/v3/projects/b659b79132d64a5aa06c9e6bd130f79c` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:46.013803 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b659b79132d64a5aa06c9e6bd130f79c)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:46.016858 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:46.067348 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:46.072351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b659b79132d64a5aa06c9e6bd130f79c', 'request_id': 'req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:46.080466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b659b79132d64a5aa06c9e6bd130f79c', 'request_id': 'req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:46.089413 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b659b79132d64a5aa06c9e6bd130f79c', 'request_id': 'req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:46.100665 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051 admin admin] The token cache is being invalidate because project b659b79132d64a5aa06c9e6bd130f79c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:52:46.100952 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:46.101671 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:46.103268 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1209/2427] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:52:45 2022] DELETE /identity/v3/projects/b659b79132d64a5aa06c9e6bd130f79c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:46.108636 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ad0cc-6d65-4106-9dec-1ccf6bc200aa service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:46.122078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ad0cc-6d65-4106-9dec-1ccf6bc200aa service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:46.187637 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f226bf-e628-4769-8894-32f46f74013b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:46.188369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f226bf-e628-4769-8894-32f46f74013b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:46.188671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f226bf-e628-4769-8894-32f46f74013b admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:46.188948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f226bf-e628-4769-8894-32f46f74013b admin admin] PATH_INFO: `/v3/users/a1477ed537e042bc99da2b2ab89a3c5f` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:46.190231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f226bf-e628-4769-8894-32f46f74013b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1477ed537e042bc99da2b2ab89a3c5f)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:46.192176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f226bf-e628-4769-8894-32f46f74013b admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:46.246561 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-16f226bf-e628-4769-8894-32f46f74013b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1477ed537e042bc99da2b2ab89a3c5f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:46.254985 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-16f226bf-e628-4769-8894-32f46f74013b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1477ed537e042bc99da2b2ab89a3c5f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:46.255606 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-16f226bf-e628-4769-8894-32f46f74013b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1477ed537e042bc99da2b2ab89a3c5f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:46.256195 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-16f226bf-e628-4769-8894-32f46f74013b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1477ed537e042bc99da2b2ab89a3c5f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:46.263527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-16f226bf-e628-4769-8894-32f46f74013b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1477ed537e042bc99da2b2ab89a3c5f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:46.285505 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1219/2428] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:52:46 2022] DELETE /identity/v3/users/a1477ed537e042bc99da2b2ab89a3c5f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:46.613629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:46.616495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:46.647434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2331159-9a7c-4d15-aa5d-b5e0eac8f051 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:46.727261 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5328d-7ac6-4196-a0d2-4e5514b4fd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:46.727898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5328d-7ac6-4196-a0d2-4e5514b4fd7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:46.728214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5328d-7ac6-4196-a0d2-4e5514b4fd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:46.728533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5328d-7ac6-4196-a0d2-4e5514b4fd7b admin admin] PATH_INFO: `/v3/projects/5e18e0d1e77141729181e1f520adeedc` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:46.729960 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5328d-7ac6-4196-a0d2-4e5514b4fd7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e18e0d1e77141729181e1f520adeedc)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:46.732349 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5328d-7ac6-4196-a0d2-4e5514b4fd7b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:46.787449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-68d5328d-7ac6-4196-a0d2-4e5514b4fd7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e18e0d1e77141729181e1f520adeedc', 'request_id': 'req-68d5328d-7ac6-4196-a0d2-4e5514b4fd7b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:46.795010 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-68d5328d-7ac6-4196-a0d2-4e5514b4fd7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e18e0d1e77141729181e1f520adeedc', 'request_id': 'req-68d5328d-7ac6-4196-a0d2-4e5514b4fd7b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:46.803930 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-68d5328d-7ac6-4196-a0d2-4e5514b4fd7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e18e0d1e77141729181e1f520adeedc', 'request_id': 'req-68d5328d-7ac6-4196-a0d2-4e5514b4fd7b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:46.816692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-68d5328d-7ac6-4196-a0d2-4e5514b4fd7b admin admin] The token cache is being invalidate because project 5e18e0d1e77141729181e1f520adeedc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:52:46.817185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-68d5328d-7ac6-4196-a0d2-4e5514b4fd7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:46.818074 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-68d5328d-7ac6-4196-a0d2-4e5514b4fd7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:46.819792 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1210/2429] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:52:46 2022] DELETE /identity/v3/projects/5e18e0d1e77141729181e1f520adeedc => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:46.824983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdd3af-a879-4699-9e88-7aea14203425 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:46.826045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdd3af-a879-4699-9e88-7aea14203425 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:46.826045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdd3af-a879-4699-9e88-7aea14203425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:46.846805 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-47bdd3af-a879-4699-9e88-7aea14203425 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:46.848068 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-47bdd3af-a879-4699-9e88-7aea14203425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:46.901918 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:46.920871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1220/2430] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:52:46.929210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7e86c-2be0-45bd-9819-a18d9d232ac3 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:46.929503 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7e86c-2be0-45bd-9819-a18d9d232ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:46.929782 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7e86c-2be0-45bd-9819-a18d9d232ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:46.954770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-59e7e86c-2be0-45bd-9819-a18d9d232ac3 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:46.955995 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-59e7e86c-2be0-45bd-9819-a18d9d232ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:46.964843 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1211/2431] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:46.969842 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bdd3af-a879-4699-9e88-7aea14203425 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:46.980396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bdd3af-a879-4699-9e88-7aea14203425 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:46.985722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2fffc-48d8-4757-afde-0508af7dacbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:46.986345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2fffc-48d8-4757-afde-0508af7dacbc admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:46.986616 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2fffc-48d8-4757-afde-0508af7dacbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:46.986896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2fffc-48d8-4757-afde-0508af7dacbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:46.987547 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2fffc-48d8-4757-afde-0508af7dacbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:52:46.987880 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2fffc-48d8-4757-afde-0508af7dacbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:46.989551 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2fffc-48d8-4757-afde-0508af7dacbc admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:47.002119 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1221/2432] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:52:47.007131 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7e86c-2be0-45bd-9819-a18d9d232ac3 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:47.018571 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7e86c-2be0-45bd-9819-a18d9d232ac3 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:47.023978 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d102a-7138-41c6-b7d7-d04259453abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:47.024580 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d102a-7138-41c6-b7d7-d04259453abb admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:47.024916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d102a-7138-41c6-b7d7-d04259453abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:47.025201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d102a-7138-41c6-b7d7-d04259453abb admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:47.025834 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d102a-7138-41c6-b7d7-d04259453abb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:47.027737 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d102a-7138-41c6-b7d7-d04259453abb admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:47.058748 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1212/2433] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:52:47 2022] POST /identity/v3/projects => generated 407 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:52:47.063570 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2fffc-48d8-4757-afde-0508af7dacbc admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:47.074923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2fffc-48d8-4757-afde-0508af7dacbc admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:47.140823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56849c-9ef9-415d-a795-63eb04b33c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:47.141464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56849c-9ef9-415d-a795-63eb04b33c01 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:47.141739 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56849c-9ef9-415d-a795-63eb04b33c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:47.142001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56849c-9ef9-415d-a795-63eb04b33c01 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:47.142629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56849c-9ef9-415d-a795-63eb04b33c01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:47.144525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56849c-9ef9-415d-a795-63eb04b33c01 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:47.171026 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1222/2434] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:52:47 2022] POST /identity/v3/users => generated 304 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:52:47.175438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d102a-7138-41c6-b7d7-d04259453abb admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:47.188285 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d102a-7138-41c6-b7d7-d04259453abb admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:47.194701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf7ae5-290b-42f2-b773-2d3305008247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:47.195323 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf7ae5-290b-42f2-b773-2d3305008247 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:47.195599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf7ae5-290b-42f2-b773-2d3305008247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:47.195850 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf7ae5-290b-42f2-b773-2d3305008247 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:47.196503 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf7ae5-290b-42f2-b773-2d3305008247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:47.198154 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf7ae5-290b-42f2-b773-2d3305008247 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:47.210675 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1213/2435] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:52:47.215165 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56849c-9ef9-415d-a795-63eb04b33c01 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:47.227948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56849c-9ef9-415d-a795-63eb04b33c01 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:47.234223 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d957c-fce1-44c2-a6af-65519f5f6996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:47.235019 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d957c-fce1-44c2-a6af-65519f5f6996 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:47.235298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d957c-fce1-44c2-a6af-65519f5f6996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:47.235574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d957c-fce1-44c2-a6af-65519f5f6996 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:47.236238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d957c-fce1-44c2-a6af-65519f5f6996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:47.237922 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d957c-fce1-44c2-a6af-65519f5f6996 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:47.250399 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1223/2436] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:52:47.256965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf7ae5-290b-42f2-b773-2d3305008247 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:47.267884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf7ae5-290b-42f2-b773-2d3305008247 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:47.273918 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feba934-3d37-44ec-b102-1520313fdaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:47.274579 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feba934-3d37-44ec-b102-1520313fdaae admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:47.274856 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feba934-3d37-44ec-b102-1520313fdaae admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:47.275126 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feba934-3d37-44ec-b102-1520313fdaae admin admin] PATH_INFO: `/v3/projects/d8d5e66c4f5c448ba71ed37cd29c5dfa/users/b597ace0ae92466782a8d2ec3aba54c7/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:47.291493 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feba934-3d37-44ec-b102-1520313fdaae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8d5e66c4f5c448ba71ed37cd29c5dfa, user_id=b597ace0ae92466782a8d2ec3aba54c7, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:47.294220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feba934-3d37-44ec-b102-1520313fdaae admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:47.306307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:47.314861 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1214/2437] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:52:47 2022] PUT /identity/v3/projects/d8d5e66c4f5c448ba71ed37cd29c5dfa/users/b597ace0ae92466782a8d2ec3aba54c7/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:47.328088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d649ea-097f-4db2-8c98-0a7a81106268 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:47.328622 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d649ea-097f-4db2-8c98-0a7a81106268 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:47.328889 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d649ea-097f-4db2-8c98-0a7a81106268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:47.358090 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-a0d649ea-097f-4db2-8c98-0a7a81106268 None None] MFA Rules not processed for user `b597ace0ae92466782a8d2ec3aba54c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:47.359285 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-a0d649ea-097f-4db2-8c98-0a7a81106268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:47.426295 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1224/2438] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:47 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:47.431490 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feba934-3d37-44ec-b102-1520313fdaae admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:47.499232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feba934-3d37-44ec-b102-1520313fdaae admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:47.504019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cfb242-78f2-401c-9341-5b69ef01e882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:47.504621 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfb242-78f2-401c-9341-5b69ef01e882 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:47.504922 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfb242-78f2-401c-9341-5b69ef01e882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:47.505212 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfb242-78f2-401c-9341-5b69ef01e882 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:47.505840 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cfb242-78f2-401c-9341-5b69ef01e882 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:47.507743 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cfb242-78f2-401c-9341-5b69ef01e882 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:47.545393 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1215/2439] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:52:47 2022] POST /identity/v3/projects => generated 407 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:52:47.550519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d649ea-097f-4db2-8c98-0a7a81106268 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:47.562656 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d649ea-097f-4db2-8c98-0a7a81106268 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:47.610500 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:47.628333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2fd5e-7c57-445c-80c6-01dbb516a511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:47.629065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2fd5e-7c57-445c-80c6-01dbb516a511 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:47.629372 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2fd5e-7c57-445c-80c6-01dbb516a511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:47.629648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2fd5e-7c57-445c-80c6-01dbb516a511 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:47.630284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2fd5e-7c57-445c-80c6-01dbb516a511 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:47.632213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2fd5e-7c57-445c-80c6-01dbb516a511 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:47.655392 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1225/2440] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:52:47 2022] POST /identity/v3/users => generated 310 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:52:47.660264 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cfb242-78f2-401c-9341-5b69ef01e882 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:47.675435 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cfb242-78f2-401c-9341-5b69ef01e882 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:47.682496 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d3397-1cdd-43e1-a141-cc782c102f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:47.683355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d3397-1cdd-43e1-a141-cc782c102f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:47.683645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d3397-1cdd-43e1-a141-cc782c102f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:47.683914 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d3397-1cdd-43e1-a141-cc782c102f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:47.684726 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d3397-1cdd-43e1-a141-cc782c102f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:47.686616 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d3397-1cdd-43e1-a141-cc782c102f2b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:47.698420 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1216/2441] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:52:47.703415 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2fd5e-7c57-445c-80c6-01dbb516a511 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:47.720333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2fd5e-7c57-445c-80c6-01dbb516a511 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:47.726140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4a7b2-9433-4e34-9281-b0e24b69e4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:47.727010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4a7b2-9433-4e34-9281-b0e24b69e4ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:47.727230 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4a7b2-9433-4e34-9281-b0e24b69e4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:47.727519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4a7b2-9433-4e34-9281-b0e24b69e4ea admin admin] PATH_INFO: `/v3/projects/b6d194d0fb844e099fd6e1c5d817264e/users/e518c7b4433747f8b191ffc6b8db28b1/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:47.745286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4a7b2-9433-4e34-9281-b0e24b69e4ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6d194d0fb844e099fd6e1c5d817264e, user_id=e518c7b4433747f8b191ffc6b8db28b1, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:47.747995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4a7b2-9433-4e34-9281-b0e24b69e4ea admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:47.764794 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1226/2442] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:52:47 2022] PUT /identity/v3/projects/b6d194d0fb844e099fd6e1c5d817264e/users/e518c7b4433747f8b191ffc6b8db28b1/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:47.769335 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d3397-1cdd-43e1-a141-cc782c102f2b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:47.780448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d3397-1cdd-43e1-a141-cc782c102f2b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:47.848871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ecf6e-a4c9-440c-9466-e562f3043a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:47.849539 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ecf6e-a4c9-440c-9466-e562f3043a16 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:47.849848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ecf6e-a4c9-440c-9466-e562f3043a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:47.850160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ecf6e-a4c9-440c-9466-e562f3043a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:47.850875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ecf6e-a4c9-440c-9466-e562f3043a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:47.852646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ecf6e-a4c9-440c-9466-e562f3043a16 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:47.865067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1217/2443] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:52:47.870057 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4a7b2-9433-4e34-9281-b0e24b69e4ea admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:47.882740 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4a7b2-9433-4e34-9281-b0e24b69e4ea admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:47.888634 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67a08d-c496-4431-b40d-008aa6ea16ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:47.889330 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67a08d-c496-4431-b40d-008aa6ea16ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:47.889613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67a08d-c496-4431-b40d-008aa6ea16ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:47.889883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67a08d-c496-4431-b40d-008aa6ea16ce admin admin] PATH_INFO: `/v3/projects/b6d194d0fb844e099fd6e1c5d817264e/users/e518c7b4433747f8b191ffc6b8db28b1/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:47.892149 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67a08d-c496-4431-b40d-008aa6ea16ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6d194d0fb844e099fd6e1c5d817264e, user_id=e518c7b4433747f8b191ffc6b8db28b1, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:47.894856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67a08d-c496-4431-b40d-008aa6ea16ce admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:47.906114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1227/2444] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:52:47 2022] PUT /identity/v3/projects/b6d194d0fb844e099fd6e1c5d817264e/users/e518c7b4433747f8b191ffc6b8db28b1/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:47.918443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7309d-70ba-4e6c-bd47-ead48e08cf3e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:47.918782 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7309d-70ba-4e6c-bd47-ead48e08cf3e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:47.919118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7309d-70ba-4e6c-bd47-ead48e08cf3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:47.954160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-cdd7309d-70ba-4e6c-bd47-ead48e08cf3e None None] MFA Rules not processed for user `e518c7b4433747f8b191ffc6b8db28b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:47.955385 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cdd7309d-70ba-4e6c-bd47-ead48e08cf3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:48.016268 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:48.029354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1218/2445] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:47 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:48.043570 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67a08d-c496-4431-b40d-008aa6ea16ce admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:48.045569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2c67a08d-c496-4431-b40d-008aa6ea16ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:48.124846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67a08d-c496-4431-b40d-008aa6ea16ce admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:48.129383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4c1d4-7a5f-4fe4-a299-d0377c89bc05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:48.129990 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4c1d4-7a5f-4fe4-a299-d0377c89bc05 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:48.130262 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4c1d4-7a5f-4fe4-a299-d0377c89bc05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:48.130526 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4c1d4-7a5f-4fe4-a299-d0377c89bc05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:48.140347 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4c1d4-7a5f-4fe4-a299-d0377c89bc05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:48.141887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4c1d4-7a5f-4fe4-a299-d0377c89bc05 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:48.151862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1228/2446] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:52:48 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:52:48.347252 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7309d-70ba-4e6c-bd47-ead48e08cf3e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:48.358119 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7309d-70ba-4e6c-bd47-ead48e08cf3e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:48.362104 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44bb3f4-6995-44be-87f4-0491efc79ea0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:48.362728 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44bb3f4-6995-44be-87f4-0491efc79ea0 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:48.363014 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44bb3f4-6995-44be-87f4-0491efc79ea0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:48.363295 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44bb3f4-6995-44be-87f4-0491efc79ea0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:48.434428 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44bb3f4-6995-44be-87f4-0491efc79ea0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:48.435873 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44bb3f4-6995-44be-87f4-0491efc79ea0 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:48.445719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1219/2447] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:52:48 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:52:58.039526 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4c1d4-7a5f-4fe4-a299-d0377c89bc05 service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:58.108595 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4c1d4-7a5f-4fe4-a299-d0377c89bc05 service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:58.113759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64633e-db0f-45b4-a9a6-4535b30d1dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:58.114409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64633e-db0f-45b4-a9a6-4535b30d1dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:58.114694 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64633e-db0f-45b4-a9a6-4535b30d1dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:58.114960 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64633e-db0f-45b4-a9a6-4535b30d1dbe admin admin] PATH_INFO: `/v3/users/b597ace0ae92466782a8d2ec3aba54c7` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:58.116110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64633e-db0f-45b4-a9a6-4535b30d1dbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=b597ace0ae92466782a8d2ec3aba54c7)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:58.118110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64633e-db0f-45b4-a9a6-4535b30d1dbe admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:58.166925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:58.168483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-cb64633e-db0f-45b4-a9a6-4535b30d1dbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b597ace0ae92466782a8d2ec3aba54c7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:58.177671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-cb64633e-db0f-45b4-a9a6-4535b30d1dbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b597ace0ae92466782a8d2ec3aba54c7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:58.178305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-cb64633e-db0f-45b4-a9a6-4535b30d1dbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b597ace0ae92466782a8d2ec3aba54c7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:58.178913 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-cb64633e-db0f-45b4-a9a6-4535b30d1dbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b597ace0ae92466782a8d2ec3aba54c7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:58.186079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-cb64633e-db0f-45b4-a9a6-4535b30d1dbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b597ace0ae92466782a8d2ec3aba54c7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:58.203797 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1229/2448] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:52:58 2022] DELETE /identity/v3/users/b597ace0ae92466782a8d2ec3aba54c7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:58.208515 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73550ad7-21b2-44d0-9444-28be68f6a91b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:58.209062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73550ad7-21b2-44d0-9444-28be68f6a91b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:58.209062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73550ad7-21b2-44d0-9444-28be68f6a91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:58.231583 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-73550ad7-21b2-44d0-9444-28be68f6a91b None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:58.232804 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-73550ad7-21b2-44d0-9444-28be68f6a91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:58.302967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1220/2449] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:58.313268 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64633e-db0f-45b4-a9a6-4535b30d1dbe admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:58.314781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-cb64633e-db0f-45b4-a9a6-4535b30d1dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:58.375667 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64633e-db0f-45b4-a9a6-4535b30d1dbe admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:58.379708 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80364a9-47d2-4578-9184-79f710367577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:58.380344 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80364a9-47d2-4578-9184-79f710367577 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:58.380623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80364a9-47d2-4578-9184-79f710367577 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:58.381001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80364a9-47d2-4578-9184-79f710367577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:58.392129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80364a9-47d2-4578-9184-79f710367577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:58.393721 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80364a9-47d2-4578-9184-79f710367577 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:58.405584 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1230/2450] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:52:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:52:58.691762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73550ad7-21b2-44d0-9444-28be68f6a91b None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:58.693292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-73550ad7-21b2-44d0-9444-28be68f6a91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:58.708809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73550ad7-21b2-44d0-9444-28be68f6a91b None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:58.715518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4bce1-700b-44c5-9d19-39c1fe52503c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:58.718795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4bce1-700b-44c5-9d19-39c1fe52503c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:58.718795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4bce1-700b-44c5-9d19-39c1fe52503c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:58.718795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4bce1-700b-44c5-9d19-39c1fe52503c admin admin] PATH_INFO: `/v3/projects/d8d5e66c4f5c448ba71ed37cd29c5dfa` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:58.719451 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4bce1-700b-44c5-9d19-39c1fe52503c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8d5e66c4f5c448ba71ed37cd29c5dfa)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:58.721426 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4bce1-700b-44c5-9d19-39c1fe52503c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:58.782753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8ed4bce1-700b-44c5-9d19-39c1fe52503c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8d5e66c4f5c448ba71ed37cd29c5dfa', 'request_id': 'req-8ed4bce1-700b-44c5-9d19-39c1fe52503c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:58.790208 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8ed4bce1-700b-44c5-9d19-39c1fe52503c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8d5e66c4f5c448ba71ed37cd29c5dfa', 'request_id': 'req-8ed4bce1-700b-44c5-9d19-39c1fe52503c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:58.801497 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8ed4bce1-700b-44c5-9d19-39c1fe52503c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8d5e66c4f5c448ba71ed37cd29c5dfa', 'request_id': 'req-8ed4bce1-700b-44c5-9d19-39c1fe52503c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:58.814008 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8ed4bce1-700b-44c5-9d19-39c1fe52503c admin admin] The token cache is being invalidate because project d8d5e66c4f5c448ba71ed37cd29c5dfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:52:58.814289 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8ed4bce1-700b-44c5-9d19-39c1fe52503c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:58.815087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8ed4bce1-700b-44c5-9d19-39c1fe52503c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:58.816805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1221/2451] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:52:58 2022] DELETE /identity/v3/projects/d8d5e66c4f5c448ba71ed37cd29c5dfa => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:58.823875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80364a9-47d2-4578-9184-79f710367577 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:58.841280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80364a9-47d2-4578-9184-79f710367577 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:58.928852 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c967f40-2a34-4432-b2eb-2d74ee60cdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:58.929857 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c967f40-2a34-4432-b2eb-2d74ee60cdbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:58.930338 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c967f40-2a34-4432-b2eb-2d74ee60cdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:58.930807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c967f40-2a34-4432-b2eb-2d74ee60cdbc admin admin] PATH_INFO: `/v3/users/e518c7b4433747f8b191ffc6b8db28b1` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:58.932787 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c967f40-2a34-4432-b2eb-2d74ee60cdbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e518c7b4433747f8b191ffc6b8db28b1)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:58.936515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c967f40-2a34-4432-b2eb-2d74ee60cdbc admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:59.017975 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-2c967f40-2a34-4432-b2eb-2d74ee60cdbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e518c7b4433747f8b191ffc6b8db28b1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:59.027172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-2c967f40-2a34-4432-b2eb-2d74ee60cdbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e518c7b4433747f8b191ffc6b8db28b1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:59.027983 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-2c967f40-2a34-4432-b2eb-2d74ee60cdbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e518c7b4433747f8b191ffc6b8db28b1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:59.028614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-2c967f40-2a34-4432-b2eb-2d74ee60cdbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e518c7b4433747f8b191ffc6b8db28b1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:59.037386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-2c967f40-2a34-4432-b2eb-2d74ee60cdbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e518c7b4433747f8b191ffc6b8db28b1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:59.042348 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:59.066190 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1231/2452] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:52:58 2022] DELETE /identity/v3/users/e518c7b4433747f8b191ffc6b8db28b1 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:59.378282 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4bce1-700b-44c5-9d19-39c1fe52503c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:59.379857 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8ed4bce1-700b-44c5-9d19-39c1fe52503c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:59.396710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4bce1-700b-44c5-9d19-39c1fe52503c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:59.412050 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:52:59.464568 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a33fb-a29c-4683-8b8f-aa9df6dadfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:59.465113 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a33fb-a29c-4683-8b8f-aa9df6dadfca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:59.465419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a33fb-a29c-4683-8b8f-aa9df6dadfca admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:59.465698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a33fb-a29c-4683-8b8f-aa9df6dadfca admin admin] PATH_INFO: `/v3/projects/b6d194d0fb844e099fd6e1c5d817264e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:59.466927 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a33fb-a29c-4683-8b8f-aa9df6dadfca admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6d194d0fb844e099fd6e1c5d817264e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:59.468859 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a33fb-a29c-4683-8b8f-aa9df6dadfca admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:59.520039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bc5a33fb-a29c-4683-8b8f-aa9df6dadfca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6d194d0fb844e099fd6e1c5d817264e', 'request_id': 'req-bc5a33fb-a29c-4683-8b8f-aa9df6dadfca'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:59.527124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bc5a33fb-a29c-4683-8b8f-aa9df6dadfca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6d194d0fb844e099fd6e1c5d817264e', 'request_id': 'req-bc5a33fb-a29c-4683-8b8f-aa9df6dadfca'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:59.536332 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bc5a33fb-a29c-4683-8b8f-aa9df6dadfca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6d194d0fb844e099fd6e1c5d817264e', 'request_id': 'req-bc5a33fb-a29c-4683-8b8f-aa9df6dadfca'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:59.548443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bc5a33fb-a29c-4683-8b8f-aa9df6dadfca admin admin] The token cache is being invalidate because project b6d194d0fb844e099fd6e1c5d817264e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:52:59.548728 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bc5a33fb-a29c-4683-8b8f-aa9df6dadfca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:59.549438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-bc5a33fb-a29c-4683-8b8f-aa9df6dadfca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:52:59.551010 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1222/2453] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:52:59 2022] DELETE /identity/v3/projects/b6d194d0fb844e099fd6e1c5d817264e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:52:59.556215 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d933b-ff78-447c-89da-2440657894ad None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:59.556756 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d933b-ff78-447c-89da-2440657894ad None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:59.557039 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d933b-ff78-447c-89da-2440657894ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:59.578972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-0c9d933b-ff78-447c-89da-2440657894ad None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:59.580226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-0c9d933b-ff78-447c-89da-2440657894ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:59.641773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1232/2454] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:52:59.651288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f246bd-73de-4a21-9e07-42f0e12e6544 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:59.651674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f246bd-73de-4a21-9e07-42f0e12e6544 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:59.651998 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f246bd-73de-4a21-9e07-42f0e12e6544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:59.680293 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-10f246bd-73de-4a21-9e07-42f0e12e6544 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:52:59.681565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-10f246bd-73de-4a21-9e07-42f0e12e6544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:52:59.690889 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1223/2455] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:52:59.695264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d933b-ff78-447c-89da-2440657894ad None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:59.707481 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d933b-ff78-447c-89da-2440657894ad None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:59.713546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96a2e1-c9f0-446b-8d13-91b23931004e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:59.714150 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96a2e1-c9f0-446b-8d13-91b23931004e admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:59.714427 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96a2e1-c9f0-446b-8d13-91b23931004e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:59.714750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96a2e1-c9f0-446b-8d13-91b23931004e admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:59.715349 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96a2e1-c9f0-446b-8d13-91b23931004e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:52:59.715643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96a2e1-c9f0-446b-8d13-91b23931004e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:59.717311 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96a2e1-c9f0-446b-8d13-91b23931004e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:59.730222 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1233/2456] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:52:59.735161 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f246bd-73de-4a21-9e07-42f0e12e6544 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:59.745875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f246bd-73de-4a21-9e07-42f0e12e6544 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:59.754728 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e286f-fecf-4c18-aa26-462678fa5649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:59.755355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e286f-fecf-4c18-aa26-462678fa5649 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:59.755637 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e286f-fecf-4c18-aa26-462678fa5649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:59.755915 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e286f-fecf-4c18-aa26-462678fa5649 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:59.757674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e286f-fecf-4c18-aa26-462678fa5649 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:59.759946 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e286f-fecf-4c18-aa26-462678fa5649 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:59.792467 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1224/2457] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:52:59 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:52:59.797282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96a2e1-c9f0-446b-8d13-91b23931004e admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:59.812273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96a2e1-c9f0-446b-8d13-91b23931004e admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:59.879216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166ff80-95d6-491d-bba1-fb5655a46d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:59.879841 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166ff80-95d6-491d-bba1-fb5655a46d00 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:59.880105 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166ff80-95d6-491d-bba1-fb5655a46d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:59.880375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166ff80-95d6-491d-bba1-fb5655a46d00 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:59.881015 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166ff80-95d6-491d-bba1-fb5655a46d00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:59.882980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166ff80-95d6-491d-bba1-fb5655a46d00 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:59.907690 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1234/2458] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:52:59 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:52:59.911968 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e286f-fecf-4c18-aa26-462678fa5649 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:59.926394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e286f-fecf-4c18-aa26-462678fa5649 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:59.932712 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec23df-1c64-45ba-9051-bb11b3df68a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:59.933356 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec23df-1c64-45ba-9051-bb11b3df68a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:59.934293 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec23df-1c64-45ba-9051-bb11b3df68a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:59.934566 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec23df-1c64-45ba-9051-bb11b3df68a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:59.935218 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec23df-1c64-45ba-9051-bb11b3df68a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:59.936820 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec23df-1c64-45ba-9051-bb11b3df68a9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:59.947925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1225/2459] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:52:59.953226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166ff80-95d6-491d-bba1-fb5655a46d00 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:52:59.968188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166ff80-95d6-491d-bba1-fb5655a46d00 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:52:59.977776 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315fc36-fc07-4031-afd0-54ce18761480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:52:59.978669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315fc36-fc07-4031-afd0-54ce18761480 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:52:59.979017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315fc36-fc07-4031-afd0-54ce18761480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:52:59.979120 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315fc36-fc07-4031-afd0-54ce18761480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:52:59.980291 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315fc36-fc07-4031-afd0-54ce18761480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:52:59.982932 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315fc36-fc07-4031-afd0-54ce18761480 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:52:59.996262 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1235/2460] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:53:00.001998 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec23df-1c64-45ba-9051-bb11b3df68a9 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:53:00.012556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec23df-1c64-45ba-9051-bb11b3df68a9 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:53:00.017712 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf99d03-f3b4-4fc6-9727-50d6a3799f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:53:00.018371 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf99d03-f3b4-4fc6-9727-50d6a3799f37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:53:00.018629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf99d03-f3b4-4fc6-9727-50d6a3799f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:53:00.019020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf99d03-f3b4-4fc6-9727-50d6a3799f37 admin admin] PATH_INFO: `/v3/projects/fecf64e1de1949a586b759ce5480e7b5/users/7f27e8bc4ef6457496d708689d7616b0/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:53:00.035447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf99d03-f3b4-4fc6-9727-50d6a3799f37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fecf64e1de1949a586b759ce5480e7b5, user_id=7f27e8bc4ef6457496d708689d7616b0, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:53:00.038232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf99d03-f3b4-4fc6-9727-50d6a3799f37 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:53:00.052957 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1226/2461] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:53:00 2022] PUT /identity/v3/projects/fecf64e1de1949a586b759ce5480e7b5/users/7f27e8bc4ef6457496d708689d7616b0/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:53:00.123299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d75f6-daa0-4165-b9d3-c8ece10b86a0 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:53:00.124110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d75f6-daa0-4165-b9d3-c8ece10b86a0 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:53:00.124110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d75f6-daa0-4165-b9d3-c8ece10b86a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:53:00.160093 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-5b5d75f6-daa0-4165-b9d3-c8ece10b86a0 None None] MFA Rules not processed for user `7f27e8bc4ef6457496d708689d7616b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:53:00.160093 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5b5d75f6-daa0-4165-b9d3-c8ece10b86a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:53:00.216045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:53:00.244514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1236/2462] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:53:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:53:00.255924 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf99d03-f3b4-4fc6-9727-50d6a3799f37 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:53:00.257667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-adf99d03-f3b4-4fc6-9727-50d6a3799f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:53:00.314235 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:53:00.329871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf99d03-f3b4-4fc6-9727-50d6a3799f37 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:53:00.333637 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0fb10-6fb6-48a6-9f87-02de5c160d6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:53:00.334289 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0fb10-6fb6-48a6-9f87-02de5c160d6f service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:53:00.334579 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0fb10-6fb6-48a6-9f87-02de5c160d6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:53:00.334877 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0fb10-6fb6-48a6-9f87-02de5c160d6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:53:00.344697 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0fb10-6fb6-48a6-9f87-02de5c160d6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:53:00.346258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0fb10-6fb6-48a6-9f87-02de5c160d6f service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:53:00.356054 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1227/2463] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:53:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:53:03.512877 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d75f6-daa0-4165-b9d3-c8ece10b86a0 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:53:03.579804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d75f6-daa0-4165-b9d3-c8ece10b86a0 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:53:03.585551 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ec34d-c7ff-4ddd-be41-6af393e1e777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:53:03.586259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ec34d-c7ff-4ddd-be41-6af393e1e777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:53:03.586536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ec34d-c7ff-4ddd-be41-6af393e1e777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:53:03.586809 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ec34d-c7ff-4ddd-be41-6af393e1e777 admin admin] PATH_INFO: `/v3/users/7f27e8bc4ef6457496d708689d7616b0` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:53:03.587970 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ec34d-c7ff-4ddd-be41-6af393e1e777 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f27e8bc4ef6457496d708689d7616b0)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:53:03.589950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ec34d-c7ff-4ddd-be41-6af393e1e777 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:53:03.642466 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9e7ec34d-c7ff-4ddd-be41-6af393e1e777 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f27e8bc4ef6457496d708689d7616b0'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:53:03.661898 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9e7ec34d-c7ff-4ddd-be41-6af393e1e777 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f27e8bc4ef6457496d708689d7616b0'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:53:03.662593 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9e7ec34d-c7ff-4ddd-be41-6af393e1e777 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f27e8bc4ef6457496d708689d7616b0'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:53:03.663335 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9e7ec34d-c7ff-4ddd-be41-6af393e1e777 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f27e8bc4ef6457496d708689d7616b0'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:53:03.672995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9e7ec34d-c7ff-4ddd-be41-6af393e1e777 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f27e8bc4ef6457496d708689d7616b0'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:53:03.697571 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1237/2464] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:53:03 2022] DELETE /identity/v3/users/7f27e8bc4ef6457496d708689d7616b0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:53:03.703593 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea4938-3753-44ef-8c2b-b077a7bfd06f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:53:03.704107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea4938-3753-44ef-8c2b-b077a7bfd06f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:53:03.704107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea4938-3753-44ef-8c2b-b077a7bfd06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:53:03.724117 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-83ea4938-3753-44ef-8c2b-b077a7bfd06f None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:53:03.725361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-83ea4938-3753-44ef-8c2b-b077a7bfd06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:53:03.798050 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1228/2465] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:53:03.808367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ec34d-c7ff-4ddd-be41-6af393e1e777 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:53:03.809954 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9e7ec34d-c7ff-4ddd-be41-6af393e1e777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:53:03.874197 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ec34d-c7ff-4ddd-be41-6af393e1e777 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:53:03.878722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7507c3f-b53a-4fe5-9888-073e9582dd0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:53:03.879331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7507c3f-b53a-4fe5-9888-073e9582dd0e service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:53:03.879603 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7507c3f-b53a-4fe5-9888-073e9582dd0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:53:03.879869 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7507c3f-b53a-4fe5-9888-073e9582dd0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:53:03.891688 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7507c3f-b53a-4fe5-9888-073e9582dd0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:53:03.893150 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7507c3f-b53a-4fe5-9888-073e9582dd0e service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:53:03.902747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1238/2466] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:53:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:53:04.184880 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea4938-3753-44ef-8c2b-b077a7bfd06f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:53:04.187077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-83ea4938-3753-44ef-8c2b-b077a7bfd06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:53:04.208342 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea4938-3753-44ef-8c2b-b077a7bfd06f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:53:04.222565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2b2e1-bba9-46c9-a174-6e346df2f02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:53:04.223296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2b2e1-bba9-46c9-a174-6e346df2f02b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:53:04.223569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2b2e1-bba9-46c9-a174-6e346df2f02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:53:04.223841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2b2e1-bba9-46c9-a174-6e346df2f02b admin admin] PATH_INFO: `/v3/projects/fecf64e1de1949a586b759ce5480e7b5` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:53:04.225167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2b2e1-bba9-46c9-a174-6e346df2f02b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fecf64e1de1949a586b759ce5480e7b5)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:53:04.227155 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2b2e1-bba9-46c9-a174-6e346df2f02b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:53:04.282769 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-72f2b2e1-bba9-46c9-a174-6e346df2f02b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fecf64e1de1949a586b759ce5480e7b5', 'request_id': 'req-72f2b2e1-bba9-46c9-a174-6e346df2f02b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:53:04.289976 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-72f2b2e1-bba9-46c9-a174-6e346df2f02b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fecf64e1de1949a586b759ce5480e7b5', 'request_id': 'req-72f2b2e1-bba9-46c9-a174-6e346df2f02b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:53:04.299259 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-72f2b2e1-bba9-46c9-a174-6e346df2f02b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fecf64e1de1949a586b759ce5480e7b5', 'request_id': 'req-72f2b2e1-bba9-46c9-a174-6e346df2f02b'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:53:04.310875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-72f2b2e1-bba9-46c9-a174-6e346df2f02b admin admin] The token cache is being invalidate because project fecf64e1de1949a586b759ce5480e7b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:53:04.311186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-72f2b2e1-bba9-46c9-a174-6e346df2f02b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:53:04.311821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-72f2b2e1-bba9-46c9-a174-6e346df2f02b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:53:04.313324 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1229/2467] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:53:04 2022] DELETE /identity/v3/projects/fecf64e1de1949a586b759ce5480e7b5 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:53:13.898510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:53:14.307539 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=61,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:53:59.123989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df998614-38ec-4d43-9abf-db2529affc9c None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:53:59.124530 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df998614-38ec-4d43-9abf-db2529affc9c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:53:59.124626 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df998614-38ec-4d43-9abf-db2529affc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:53:59.144432 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-df998614-38ec-4d43-9abf-db2529affc9c None None] MFA Rules not processed for user `bfaf6a2b8a2348b4aa603cda5fe624f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:53:59.145683 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-df998614-38ec-4d43-9abf-db2529affc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:53:59.203125 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1239/2468] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:53:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:54:07.029068 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2b2e1-bba9-46c9-a174-6e346df2f02b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:07.030797 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-72f2b2e1-bba9-46c9-a174-6e346df2f02b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:07.131638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2b2e1-bba9-46c9-a174-6e346df2f02b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:07.136089 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7deb277-2b00-4666-afae-0fed8385b7b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:07.136828 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7deb277-2b00-4666-afae-0fed8385b7b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:07.137295 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7deb277-2b00-4666-afae-0fed8385b7b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:07.137568 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7deb277-2b00-4666-afae-0fed8385b7b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:07.139731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d7deb277-2b00-4666-afae-0fed8385b7b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:07.218014 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7deb277-2b00-4666-afae-0fed8385b7b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:07.219510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7deb277-2b00-4666-afae-0fed8385b7b9 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:07.238011 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1230/2469] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:54:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:54:08.090771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc79a7-7c07-455d-8d5e-84c888161d8d None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:08.091356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc79a7-7c07-455d-8d5e-84c888161d8d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:08.091356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc79a7-7c07-455d-8d5e-84c888161d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:08.112767 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-04fc79a7-7c07-455d-8d5e-84c888161d8d None None] MFA Rules not processed for user `0216bf67389e4627bf953b7dcc97a88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:54:08.114031 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-04fc79a7-7c07-455d-8d5e-84c888161d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:08.176562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1240/2470] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 26 05:54:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:54:08.233573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7deb277-2b00-4666-afae-0fed8385b7b9 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:08.233573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d7deb277-2b00-4666-afae-0fed8385b7b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:08.309311 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7deb277-2b00-4666-afae-0fed8385b7b9 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:08.314550 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb933b4b-d841-4cba-9398-ed890bdf3e02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f258aba84d4dd59ebe8de973ac65c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:08.315191 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb933b4b-d841-4cba-9398-ed890bdf3e02 service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:08.315489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb933b4b-d841-4cba-9398-ed890bdf3e02 service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:08.315755 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb933b4b-d841-4cba-9398-ed890bdf3e02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:08.326097 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb933b4b-d841-4cba-9398-ed890bdf3e02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:08.327786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb933b4b-d841-4cba-9398-ed890bdf3e02 service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:08.337655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1231/2471] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 26 05:54:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:54:10.618612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc79a7-7c07-455d-8d5e-84c888161d8d None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:10.620235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-04fc79a7-7c07-455d-8d5e-84c888161d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:10.634577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc79a7-7c07-455d-8d5e-84c888161d8d None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:10.642984 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4bcd38-bdf8-4c66-be32-bc7c27756167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:10.644367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bcd38-bdf8-4c66-be32-bc7c27756167 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:10.645086 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bcd38-bdf8-4c66-be32-bc7c27756167 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:10.645678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bcd38-bdf8-4c66-be32-bc7c27756167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:10.649310 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ce4bcd38-bdf8-4c66-be32-bc7c27756167 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:10.675341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4bcd38-bdf8-4c66-be32-bc7c27756167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:10.677163 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4bcd38-bdf8-4c66-be32-bc7c27756167 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:10.692685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1241/2472] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:54:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:54:18.333796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=86 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:54:20.685258 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=56 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:54:24.453109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b4aa0-795b-4162-9093-819d1e37caa4 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:24.453795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b4aa0-795b-4162-9093-819d1e37caa4 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:24.453795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b4aa0-795b-4162-9093-819d1e37caa4 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:24.455074 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1232/2473] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 26 05:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:54:24.461498 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62730912-b60e-4b12-a863-230ba8395ecd None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:24.462045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62730912-b60e-4b12-a863-230ba8395ecd None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:24.462045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62730912-b60e-4b12-a863-230ba8395ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:24.504715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-62730912-b60e-4b12-a863-230ba8395ecd None None] MFA Rules not processed for user `92556048cbd04c409c3f4b6fd645627f`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:54:24.507413 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-62730912-b60e-4b12-a863-230ba8395ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:24.517628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1242/2474] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 26 05:54:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:54:24.529845 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b4aa0-795b-4162-9093-819d1e37caa4 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:24.531592 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d08b4aa0-795b-4162-9093-819d1e37caa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:24.591028 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b4aa0-795b-4162-9093-819d1e37caa4 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:24.594553 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f8f2a-f03a-40a3-ad2a-689d3e42706e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c0fabecff2457e931c8c1c36681197', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:24.595186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f8f2a-f03a-40a3-ad2a-689d3e42706e service glance] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:24.595460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f8f2a-f03a-40a3-ad2a-689d3e42706e service glance] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:24.595726 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f8f2a-f03a-40a3-ad2a-689d3e42706e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:24.605975 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f8f2a-f03a-40a3-ad2a-689d3e42706e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:24.607391 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f8f2a-f03a-40a3-ad2a-689d3e42706e service glance] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:24.617835 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1233/2475] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:54:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:54:24.705562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a475fa-7628-4d00-a987-7001cffed7ae None None] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:24.705847 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a475fa-7628-4d00-a987-7001cffed7ae None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:24.706137 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a475fa-7628-4d00-a987-7001cffed7ae None None] PATH_INFO: `/` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:24.707501 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1243/2476] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 26 05:54:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:54:24.713028 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aadfad-be9b-48af-b47d-d3bbf0a54d9b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:24.713353 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aadfad-be9b-48af-b47d-d3bbf0a54d9b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:24.713665 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aadfad-be9b-48af-b47d-d3bbf0a54d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:24.752186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-34aadfad-be9b-48af-b47d-d3bbf0a54d9b None None] MFA Rules not processed for user `6bf0f594ce7646b7b6ddb8a8553509d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:54:24.753436 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-34aadfad-be9b-48af-b47d-d3bbf0a54d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:24.761123 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1234/2477] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 26 05:54:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:54:24.773513 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a475fa-7628-4d00-a987-7001cffed7ae None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:24.785075 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a475fa-7628-4d00-a987-7001cffed7ae None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:24.788916 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082653f-e214-4913-9cbd-4302611dc776 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:24.789547 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082653f-e214-4913-9cbd-4302611dc776 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:24.789858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082653f-e214-4913-9cbd-4302611dc776 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:24.790140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082653f-e214-4913-9cbd-4302611dc776 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:24.797522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082653f-e214-4913-9cbd-4302611dc776 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:24.798950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082653f-e214-4913-9cbd-4302611dc776 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:24.807017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1244/2478] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:54:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:54:24.901762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209195f-4893-4c3a-8e55-12a48d55f3c4 None None] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:24.902080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209195f-4893-4c3a-8e55-12a48d55f3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:24.902319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209195f-4893-4c3a-8e55-12a48d55f3c4 None None] PATH_INFO: `/` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:24.903722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1235/2479] 10.222.0.57 () {54 vars in 921 bytes} [Mon Sep 26 05:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 05:54:24.912192 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622ee4-8f6d-482f-aad6-4f52c76f5a0d None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:24.912581 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622ee4-8f6d-482f-aad6-4f52c76f5a0d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:24.912925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69622ee4-8f6d-482f-aad6-4f52c76f5a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:24.937970 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-69622ee4-8f6d-482f-aad6-4f52c76f5a0d None None] MFA Rules not processed for user `92556048cbd04c409c3f4b6fd645627f`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:54:24.939190 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-69622ee4-8f6d-482f-aad6-4f52c76f5a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:24.951178 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1245/2480] 10.222.0.57 () {58 vars in 1018 bytes} [Mon Sep 26 05:54:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:54:24.964520 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209195f-4893-4c3a-8e55-12a48d55f3c4 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:24.982549 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209195f-4893-4c3a-8e55-12a48d55f3c4 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:24.988347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671ba47-dadc-467b-9a5a-9501876485a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:24.988947 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671ba47-dadc-467b-9a5a-9501876485a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:24.989259 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671ba47-dadc-467b-9a5a-9501876485a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:24.989527 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671ba47-dadc-467b-9a5a-9501876485a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:25.001461 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671ba47-dadc-467b-9a5a-9501876485a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:25.002985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671ba47-dadc-467b-9a5a-9501876485a9 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:25.014756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1236/2481] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:54:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:54:26.801634 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69622ee4-8f6d-482f-aad6-4f52c76f5a0d None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:26.803383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-69622ee4-8f6d-482f-aad6-4f52c76f5a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:26.818389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69622ee4-8f6d-482f-aad6-4f52c76f5a0d None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:26.825326 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f35efc-c891-46f0-afc9-0380c79d2032 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:26.825957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f35efc-c891-46f0-afc9-0380c79d2032 service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:26.826225 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f35efc-c891-46f0-afc9-0380c79d2032 service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:26.826490 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f35efc-c891-46f0-afc9-0380c79d2032 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:26.828889 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-62f35efc-c891-46f0-afc9-0380c79d2032 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:26.916644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f35efc-c891-46f0-afc9-0380c79d2032 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:26.918190 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f35efc-c891-46f0-afc9-0380c79d2032 service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:26.937081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1246/2482] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:54:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:54:27.684173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f229e-821e-460a-abf1-bdb3e81c4a6c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:27.684696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f229e-821e-460a-abf1-bdb3e81c4a6c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:27.684696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f229e-821e-460a-abf1-bdb3e81c4a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:27.713416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-a97f229e-821e-460a-abf1-bdb3e81c4a6c None None] MFA Rules not processed for user `0216bf67389e4627bf953b7dcc97a88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:54:27.714630 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a97f229e-821e-460a-abf1-bdb3e81c4a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:27.723247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1237/2483] 10.222.0.57 () {58 vars in 1009 bytes} [Mon Sep 26 05:54:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:54:27.737517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f35efc-c891-46f0-afc9-0380c79d2032 service nova] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:27.749862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f35efc-c891-46f0-afc9-0380c79d2032 service nova] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:27.755578 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeea0437-a3e9-49ca-93aa-218d26dccdc3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f258aba84d4dd59ebe8de973ac65c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:27.756191 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea0437-a3e9-49ca-93aa-218d26dccdc3 service swift] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:27.756475 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea0437-a3e9-49ca-93aa-218d26dccdc3 service swift] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:27.756745 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeea0437-a3e9-49ca-93aa-218d26dccdc3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:27.766897 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea0437-a3e9-49ca-93aa-218d26dccdc3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:27.768648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeea0437-a3e9-49ca-93aa-218d26dccdc3 service swift] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:27.778010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1247/2484] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 26 05:54:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:54:37.706836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=44 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:54:37.773986 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=65 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:54:45.836273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f229e-821e-460a-abf1-bdb3e81c4a6c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:45.846839 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f229e-821e-460a-abf1-bdb3e81c4a6c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:45.851397 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab17c9b-19a1-4245-a842-e91d38b1d7ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:45.851995 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab17c9b-19a1-4245-a842-e91d38b1d7ab service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:45.852264 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab17c9b-19a1-4245-a842-e91d38b1d7ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:45.852529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab17c9b-19a1-4245-a842-e91d38b1d7ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:45.854525 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-eab17c9b-19a1-4245-a842-e91d38b1d7ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:45.864917 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab17c9b-19a1-4245-a842-e91d38b1d7ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:45.866388 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab17c9b-19a1-4245-a842-e91d38b1d7ab service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:45.874501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1238/2485] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:54:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:54:48.072853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd58f91-da99-4baa-9eaf-70e9f8d6467e None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:48.073714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd58f91-da99-4baa-9eaf-70e9f8d6467e None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:48.073714 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd58f91-da99-4baa-9eaf-70e9f8d6467e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:48.197107 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-4fd58f91-da99-4baa-9eaf-70e9f8d6467e None None] MFA Rules not processed for user `0216bf67389e4627bf953b7dcc97a88c`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:54:48.199656 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4fd58f91-da99-4baa-9eaf-70e9f8d6467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:48.239297 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1248/2486] 10.222.0.57 () {58 vars in 967 bytes} [Mon Sep 26 05:54:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:54:48.258317 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab17c9b-19a1-4245-a842-e91d38b1d7ab service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:48.289275 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab17c9b-19a1-4245-a842-e91d38b1d7ab service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:48.298025 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c63c0f-19db-4dd2-8a0f-13271fe2be9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f258aba84d4dd59ebe8de973ac65c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:48.298359 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c63c0f-19db-4dd2-8a0f-13271fe2be9e service swift] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:48.298816 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c63c0f-19db-4dd2-8a0f-13271fe2be9e service swift] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:48.299239 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c63c0f-19db-4dd2-8a0f-13271fe2be9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:48.312638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c63c0f-19db-4dd2-8a0f-13271fe2be9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:48.315996 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c63c0f-19db-4dd2-8a0f-13271fe2be9e service swift] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:48.335193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1239/2487] 10.222.0.57 () {60 vars in 1384 bytes} [Mon Sep 26 05:54:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 05:54:54.203347 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd58f91-da99-4baa-9eaf-70e9f8d6467e None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:54.213522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd58f91-da99-4baa-9eaf-70e9f8d6467e None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:54.217724 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0d686-ef04-4eb1-943f-c36f6bc190aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:54.218952 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0d686-ef04-4eb1-943f-c36f6bc190aa service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:54.218952 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0d686-ef04-4eb1-943f-c36f6bc190aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:54.218952 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0d686-ef04-4eb1-943f-c36f6bc190aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:54.221042 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-bbf0d686-ef04-4eb1-943f-c36f6bc190aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:54.291789 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0d686-ef04-4eb1-943f-c36f6bc190aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:54.293301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0d686-ef04-4eb1-943f-c36f6bc190aa service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:54.304380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1249/2488] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:54:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:54:57.580253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c63c0f-19db-4dd2-8a0f-13271fe2be9e service swift] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:57.582920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-68c63c0f-19db-4dd2-8a0f-13271fe2be9e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:57.656463 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0d686-ef04-4eb1-943f-c36f6bc190aa service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:57.658729 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-bbf0d686-ef04-4eb1-943f-c36f6bc190aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:57.674483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0d686-ef04-4eb1-943f-c36f6bc190aa service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:57.679319 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0162ca-22b8-4331-afef-1bc807866833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:57.680146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0162ca-22b8-4331-afef-1bc807866833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:57.680473 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0162ca-22b8-4331-afef-1bc807866833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:57.680820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0162ca-22b8-4331-afef-1bc807866833 admin admin] PATH_INFO: `/v3/users/8c26eec9a5e841f8b760cc8cef309b18` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:57.687807 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c63c0f-19db-4dd2-8a0f-13271fe2be9e service swift] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:57.693983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b5bed-694c-479f-9b2f-a4e432b919c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:57.695084 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b5bed-694c-479f-9b2f-a4e432b919c8 service placement] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:57.695620 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b5bed-694c-479f-9b2f-a4e432b919c8 service placement] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:57.696262 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b5bed-694c-479f-9b2f-a4e432b919c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:57.698315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d79b5bed-694c-479f-9b2f-a4e432b919c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:57.709747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0162ca-22b8-4331-afef-1bc807866833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c26eec9a5e841f8b760cc8cef309b18)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:57.712978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0162ca-22b8-4331-afef-1bc807866833 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:57.714698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b5bed-694c-479f-9b2f-a4e432b919c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:57.716913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b5bed-694c-479f-9b2f-a4e432b919c8 service placement] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:57.734270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1240/2489] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:54:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:54:57.766641 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0b0162ca-22b8-4331-afef-1bc807866833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c26eec9a5e841f8b760cc8cef309b18'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:54:57.775750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0b0162ca-22b8-4331-afef-1bc807866833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c26eec9a5e841f8b760cc8cef309b18'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:54:57.776025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0b0162ca-22b8-4331-afef-1bc807866833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c26eec9a5e841f8b760cc8cef309b18'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:54:57.776637 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0b0162ca-22b8-4331-afef-1bc807866833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c26eec9a5e841f8b760cc8cef309b18'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:54:57.784257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0b0162ca-22b8-4331-afef-1bc807866833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c26eec9a5e841f8b760cc8cef309b18'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:54:57.802350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1250/2490] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:54:57 2022] DELETE /identity/v3/users/8c26eec9a5e841f8b760cc8cef309b18 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:54:58.003156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b5bed-694c-479f-9b2f-a4e432b919c8 service placement] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:58.015344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b5bed-694c-479f-9b2f-a4e432b919c8 service placement] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:58.077925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115cf49-3881-45d7-9291-7528668cff3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:58.078539 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115cf49-3881-45d7-9291-7528668cff3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:58.078838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115cf49-3881-45d7-9291-7528668cff3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:58.079103 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115cf49-3881-45d7-9291-7528668cff3e admin admin] PATH_INFO: `/v3/projects/9fc5a928f77549839c85f3ab7e7e13b6` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:58.089945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115cf49-3881-45d7-9291-7528668cff3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fc5a928f77549839c85f3ab7e7e13b6)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:58.091844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115cf49-3881-45d7-9291-7528668cff3e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:58.144654 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:54:58.146007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-4115cf49-3881-45d7-9291-7528668cff3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fc5a928f77549839c85f3ab7e7e13b6', 'request_id': 'req-4115cf49-3881-45d7-9291-7528668cff3e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:54:58.153701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-4115cf49-3881-45d7-9291-7528668cff3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fc5a928f77549839c85f3ab7e7e13b6', 'request_id': 'req-4115cf49-3881-45d7-9291-7528668cff3e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:54:58.162762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-4115cf49-3881-45d7-9291-7528668cff3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fc5a928f77549839c85f3ab7e7e13b6', 'request_id': 'req-4115cf49-3881-45d7-9291-7528668cff3e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:54:58.173595 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-4115cf49-3881-45d7-9291-7528668cff3e admin admin] The token cache is being invalidate because project 9fc5a928f77549839c85f3ab7e7e13b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:54:58.173797 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-4115cf49-3881-45d7-9291-7528668cff3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:54:58.174545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-4115cf49-3881-45d7-9291-7528668cff3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:54:58.176023 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1241/2491] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:54:58 2022] DELETE /identity/v3/projects/9fc5a928f77549839c85f3ab7e7e13b6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:54:58.181333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faef183-30fd-480b-a218-a717d40156d5 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:58.181621 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faef183-30fd-480b-a218-a717d40156d5 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:58.181943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faef183-30fd-480b-a218-a717d40156d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:58.201969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-6faef183-30fd-480b-a218-a717d40156d5 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:54:58.203175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6faef183-30fd-480b-a218-a717d40156d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:58.214267 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:54:58.270745 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1251/2492] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:54:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:54:58.280260 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa969c6-7318-4b74-a5a2-6953a9caf440 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:58.280543 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa969c6-7318-4b74-a5a2-6953a9caf440 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:58.280815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa969c6-7318-4b74-a5a2-6953a9caf440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:58.301059 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-dfa969c6-7318-4b74-a5a2-6953a9caf440 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:54:58.302286 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-dfa969c6-7318-4b74-a5a2-6953a9caf440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:58.311055 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1242/2493] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:54:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:54:58.317093 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faef183-30fd-480b-a218-a717d40156d5 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:58.329501 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faef183-30fd-480b-a218-a717d40156d5 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:58.335921 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2ea73-38f2-4ac3-8305-6115e0673ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:58.336538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2ea73-38f2-4ac3-8305-6115e0673ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:58.336838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2ea73-38f2-4ac3-8305-6115e0673ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:58.337085 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2ea73-38f2-4ac3-8305-6115e0673ed0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:58.337771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2ea73-38f2-4ac3-8305-6115e0673ed0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:54:58.338077 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2ea73-38f2-4ac3-8305-6115e0673ed0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:58.339681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2ea73-38f2-4ac3-8305-6115e0673ed0 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:58.352423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1252/2494] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:54:58.357030 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa969c6-7318-4b74-a5a2-6953a9caf440 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:58.368216 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa969c6-7318-4b74-a5a2-6953a9caf440 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:58.373619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3e048-c8e1-4462-a1bf-cae91462016c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:58.374201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3e048-c8e1-4462-a1bf-cae91462016c admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:58.374492 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3e048-c8e1-4462-a1bf-cae91462016c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:58.374771 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3e048-c8e1-4462-a1bf-cae91462016c admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:58.375410 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3e048-c8e1-4462-a1bf-cae91462016c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:58.377361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3e048-c8e1-4462-a1bf-cae91462016c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:58.412032 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1243/2495] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:54:58 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:54:58.417076 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2ea73-38f2-4ac3-8305-6115e0673ed0 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:58.432661 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2ea73-38f2-4ac3-8305-6115e0673ed0 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:58.497920 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa72ea-0c66-42dd-b089-c22869ba7db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:58.498545 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa72ea-0c66-42dd-b089-c22869ba7db2 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:58.498828 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa72ea-0c66-42dd-b089-c22869ba7db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:58.499103 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa72ea-0c66-42dd-b089-c22869ba7db2 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:58.499750 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa72ea-0c66-42dd-b089-c22869ba7db2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:58.501740 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fa72ea-0c66-42dd-b089-c22869ba7db2 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:58.524575 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1253/2496] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:54:58 2022] POST /identity/v3/users => generated 284 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:54:58.529033 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3e048-c8e1-4462-a1bf-cae91462016c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:58.539705 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3e048-c8e1-4462-a1bf-cae91462016c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:58.544710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b2e4c-6f11-42ee-bf7e-d9a78aebb768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:58.545352 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b2e4c-6f11-42ee-bf7e-d9a78aebb768 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:58.545626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b2e4c-6f11-42ee-bf7e-d9a78aebb768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:58.545893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b2e4c-6f11-42ee-bf7e-d9a78aebb768 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:58.546533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b2e4c-6f11-42ee-bf7e-d9a78aebb768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:58.548119 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b2e4c-6f11-42ee-bf7e-d9a78aebb768 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:58.558340 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1244/2497] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:54:58.562472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa72ea-0c66-42dd-b089-c22869ba7db2 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:58.573080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa72ea-0c66-42dd-b089-c22869ba7db2 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:58.579191 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d4acf-3652-459a-8ca4-d839efd8fa9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:58.579796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d4acf-3652-459a-8ca4-d839efd8fa9d admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:58.580049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d4acf-3652-459a-8ca4-d839efd8fa9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:58.580303 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d4acf-3652-459a-8ca4-d839efd8fa9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:58.580944 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d4acf-3652-459a-8ca4-d839efd8fa9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:58.582925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d4acf-3652-459a-8ca4-d839efd8fa9d admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:58.594670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1254/2498] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:54:58.600644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b2e4c-6f11-42ee-bf7e-d9a78aebb768 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:58.611160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b2e4c-6f11-42ee-bf7e-d9a78aebb768 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:58.616371 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa85ef5-d5e7-44a5-971e-4e801a834a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:58.617029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa85ef5-d5e7-44a5-971e-4e801a834a87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:58.617339 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa85ef5-d5e7-44a5-971e-4e801a834a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:58.617621 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa85ef5-d5e7-44a5-971e-4e801a834a87 admin admin] PATH_INFO: `/v3/projects/3d0b1b7222454841a0af58d50e630223/users/5db01b3c0da547ce8e63a50b7be5890b/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:58.633073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa85ef5-d5e7-44a5-971e-4e801a834a87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d0b1b7222454841a0af58d50e630223, user_id=5db01b3c0da547ce8e63a50b7be5890b, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:58.635829 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa85ef5-d5e7-44a5-971e-4e801a834a87 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:58.651043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1245/2499] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:54:58 2022] PUT /identity/v3/projects/3d0b1b7222454841a0af58d50e630223/users/5db01b3c0da547ce8e63a50b7be5890b/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:54:58.662068 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a720e31-8ac1-4c70-bfac-08a5656a0c06 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:58.662629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a720e31-8ac1-4c70-bfac-08a5656a0c06 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:58.662629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a720e31-8ac1-4c70-bfac-08a5656a0c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:58.705193 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-5a720e31-8ac1-4c70-bfac-08a5656a0c06 None None] MFA Rules not processed for user `5db01b3c0da547ce8e63a50b7be5890b`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:54:58.706333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5a720e31-8ac1-4c70-bfac-08a5656a0c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:58.790410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1255/2500] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:54:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:54:58.801822 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa85ef5-d5e7-44a5-971e-4e801a834a87 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:54:58.808874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-efa85ef5-d5e7-44a5-971e-4e801a834a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:54:58.872196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa85ef5-d5e7-44a5-971e-4e801a834a87 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:54:58.875899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188b463-5895-4634-9520-b0cf247ff259 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:54:58.876518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188b463-5895-4634-9520-b0cf247ff259 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:54:58.876791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188b463-5895-4634-9520-b0cf247ff259 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:54:58.877060 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188b463-5895-4634-9520-b0cf247ff259 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:54:58.887321 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188b463-5895-4634-9520-b0cf247ff259 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:54:58.889345 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188b463-5895-4634-9520-b0cf247ff259 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:54:58.900023 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1246/2501] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:54:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:55:07.535789 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a720e31-8ac1-4c70-bfac-08a5656a0c06 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:07.549676 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:07.606703 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a720e31-8ac1-4c70-bfac-08a5656a0c06 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:07.611997 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcfff8-e2ea-46a3-9a3c-ea7e56c75d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:07.612670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcfff8-e2ea-46a3-9a3c-ea7e56c75d0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:07.612947 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcfff8-e2ea-46a3-9a3c-ea7e56c75d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:07.613256 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcfff8-e2ea-46a3-9a3c-ea7e56c75d0d admin admin] PATH_INFO: `/v3/users/5db01b3c0da547ce8e63a50b7be5890b` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:07.614467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcfff8-e2ea-46a3-9a3c-ea7e56c75d0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5db01b3c0da547ce8e63a50b7be5890b)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:07.616383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcfff8-e2ea-46a3-9a3c-ea7e56c75d0d admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:07.669493 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a5dcfff8-e2ea-46a3-9a3c-ea7e56c75d0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5db01b3c0da547ce8e63a50b7be5890b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:07.681441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a5dcfff8-e2ea-46a3-9a3c-ea7e56c75d0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5db01b3c0da547ce8e63a50b7be5890b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:07.682363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a5dcfff8-e2ea-46a3-9a3c-ea7e56c75d0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5db01b3c0da547ce8e63a50b7be5890b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:07.683094 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a5dcfff8-e2ea-46a3-9a3c-ea7e56c75d0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5db01b3c0da547ce8e63a50b7be5890b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:07.691333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a5dcfff8-e2ea-46a3-9a3c-ea7e56c75d0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5db01b3c0da547ce8e63a50b7be5890b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:07.713847 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1256/2502] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:55:07 2022] DELETE /identity/v3/users/5db01b3c0da547ce8e63a50b7be5890b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:07.718945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb5ce6-dca0-4015-b1cf-f9594cfd1ec6 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:07.719781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb5ce6-dca0-4015-b1cf-f9594cfd1ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:07.720033 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb5ce6-dca0-4015-b1cf-f9594cfd1ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:07.746612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-3bcb5ce6-dca0-4015-b1cf-f9594cfd1ec6 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:07.747842 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3bcb5ce6-dca0-4015-b1cf-f9594cfd1ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:07.811244 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1247/2503] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:55:07.822826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcfff8-e2ea-46a3-9a3c-ea7e56c75d0d admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:07.824382 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-a5dcfff8-e2ea-46a3-9a3c-ea7e56c75d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:07.885332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcfff8-e2ea-46a3-9a3c-ea7e56c75d0d admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:07.889239 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17cce2-d79f-4739-b3bd-9e6dc2a56180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:07.889860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17cce2-d79f-4739-b3bd-9e6dc2a56180 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:07.890139 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17cce2-d79f-4739-b3bd-9e6dc2a56180 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:07.890411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17cce2-d79f-4739-b3bd-9e6dc2a56180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:07.900401 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17cce2-d79f-4739-b3bd-9e6dc2a56180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:07.901898 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17cce2-d79f-4739-b3bd-9e6dc2a56180 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:07.912153 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1257/2504] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:55:08.223934 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb5ce6-dca0-4015-b1cf-f9594cfd1ec6 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:08.225489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3bcb5ce6-dca0-4015-b1cf-f9594cfd1ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:08.239096 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb5ce6-dca0-4015-b1cf-f9594cfd1ec6 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:08.245810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa3fa9-1f22-4acf-86a9-e38c293b9d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:08.246508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa3fa9-1f22-4acf-86a9-e38c293b9d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:08.246793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa3fa9-1f22-4acf-86a9-e38c293b9d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:08.247194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa3fa9-1f22-4acf-86a9-e38c293b9d29 admin admin] PATH_INFO: `/v3/projects/3d0b1b7222454841a0af58d50e630223` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:08.248644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa3fa9-1f22-4acf-86a9-e38c293b9d29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d0b1b7222454841a0af58d50e630223)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:08.250760 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa3fa9-1f22-4acf-86a9-e38c293b9d29 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:08.264498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:08.310419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5aaa3fa9-1f22-4acf-86a9-e38c293b9d29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d0b1b7222454841a0af58d50e630223', 'request_id': 'req-5aaa3fa9-1f22-4acf-86a9-e38c293b9d29'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:08.317585 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5aaa3fa9-1f22-4acf-86a9-e38c293b9d29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d0b1b7222454841a0af58d50e630223', 'request_id': 'req-5aaa3fa9-1f22-4acf-86a9-e38c293b9d29'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:08.328179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5aaa3fa9-1f22-4acf-86a9-e38c293b9d29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d0b1b7222454841a0af58d50e630223', 'request_id': 'req-5aaa3fa9-1f22-4acf-86a9-e38c293b9d29'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:08.340611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5aaa3fa9-1f22-4acf-86a9-e38c293b9d29 admin admin] The token cache is being invalidate because project 3d0b1b7222454841a0af58d50e630223 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:55:08.340901 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5aaa3fa9-1f22-4acf-86a9-e38c293b9d29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:08.341622 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5aaa3fa9-1f22-4acf-86a9-e38c293b9d29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:08.343175 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1248/2505] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:55:08 2022] DELETE /identity/v3/projects/3d0b1b7222454841a0af58d50e630223 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:08.348457 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc44aec-6a40-4858-bce0-0fbaa7298d08 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:08.348808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc44aec-6a40-4858-bce0-0fbaa7298d08 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:08.349062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc44aec-6a40-4858-bce0-0fbaa7298d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:08.371499 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-dfc44aec-6a40-4858-bce0-0fbaa7298d08 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:08.372802 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-dfc44aec-6a40-4858-bce0-0fbaa7298d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:08.406987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:08.445337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1258/2506] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:55:08.453704 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88256067-92f3-4ac7-a376-934d5c4a1836 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:08.454225 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88256067-92f3-4ac7-a376-934d5c4a1836 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:08.454690 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88256067-92f3-4ac7-a376-934d5c4a1836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:08.480906 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-88256067-92f3-4ac7-a376-934d5c4a1836 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:08.482297 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-88256067-92f3-4ac7-a376-934d5c4a1836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:08.491036 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1249/2507] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:55:08.495751 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc44aec-6a40-4858-bce0-0fbaa7298d08 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:08.506976 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc44aec-6a40-4858-bce0-0fbaa7298d08 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:08.514016 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf87a51-8c0d-4a2d-a13c-56e63e84f23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:08.514661 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf87a51-8c0d-4a2d-a13c-56e63e84f23d admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:08.514941 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf87a51-8c0d-4a2d-a13c-56e63e84f23d admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:08.515218 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf87a51-8c0d-4a2d-a13c-56e63e84f23d admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:08.515891 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf87a51-8c0d-4a2d-a13c-56e63e84f23d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:55:08.516198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf87a51-8c0d-4a2d-a13c-56e63e84f23d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:08.518060 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf87a51-8c0d-4a2d-a13c-56e63e84f23d admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:08.531823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1259/2508] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:55:08.536913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88256067-92f3-4ac7-a376-934d5c4a1836 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:08.547849 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88256067-92f3-4ac7-a376-934d5c4a1836 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:08.553270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8b642-3f39-4205-8b2b-80faadd5ba59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:08.553935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8b642-3f39-4205-8b2b-80faadd5ba59 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:08.554167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8b642-3f39-4205-8b2b-80faadd5ba59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:08.554433 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8b642-3f39-4205-8b2b-80faadd5ba59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:08.555107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8b642-3f39-4205-8b2b-80faadd5ba59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:08.557182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8b642-3f39-4205-8b2b-80faadd5ba59 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:08.597260 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1250/2509] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:55:08 2022] POST /identity/v3/projects => generated 405 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:55:08.598813 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf87a51-8c0d-4a2d-a13c-56e63e84f23d admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:08.615383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf87a51-8c0d-4a2d-a13c-56e63e84f23d admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:08.694607 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b19a6f-2593-4b35-bd57-641eb3dbadf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:08.695233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b19a6f-2593-4b35-bd57-641eb3dbadf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:08.695510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b19a6f-2593-4b35-bd57-641eb3dbadf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:08.695784 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b19a6f-2593-4b35-bd57-641eb3dbadf5 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:08.696435 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b19a6f-2593-4b35-bd57-641eb3dbadf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:08.698513 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b19a6f-2593-4b35-bd57-641eb3dbadf5 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:08.723746 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1260/2510] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:55:08 2022] POST /identity/v3/users => generated 303 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:55:08.729389 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8b642-3f39-4205-8b2b-80faadd5ba59 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:08.741021 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8b642-3f39-4205-8b2b-80faadd5ba59 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:08.746295 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27848d-052b-4084-9a5b-cb12696b67e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:08.746930 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27848d-052b-4084-9a5b-cb12696b67e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:08.747205 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27848d-052b-4084-9a5b-cb12696b67e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:08.747479 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27848d-052b-4084-9a5b-cb12696b67e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:08.748137 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27848d-052b-4084-9a5b-cb12696b67e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:08.749830 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27848d-052b-4084-9a5b-cb12696b67e7 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:08.761443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1251/2511] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:55:08.765894 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b19a6f-2593-4b35-bd57-641eb3dbadf5 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:08.777664 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b19a6f-2593-4b35-bd57-641eb3dbadf5 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:08.783547 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4314299-21ec-44bd-9034-4f16ff36a202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:08.784617 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4314299-21ec-44bd-9034-4f16ff36a202 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:08.785089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4314299-21ec-44bd-9034-4f16ff36a202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:08.785575 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4314299-21ec-44bd-9034-4f16ff36a202 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:08.786465 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4314299-21ec-44bd-9034-4f16ff36a202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:08.788216 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4314299-21ec-44bd-9034-4f16ff36a202 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:08.800380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1261/2512] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:55:08.804805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27848d-052b-4084-9a5b-cb12696b67e7 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:08.815750 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27848d-052b-4084-9a5b-cb12696b67e7 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:08.821453 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c6369-a9f5-44b4-91c7-8978e4b99288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:08.822153 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c6369-a9f5-44b4-91c7-8978e4b99288 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:08.822434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c6369-a9f5-44b4-91c7-8978e4b99288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:08.822741 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c6369-a9f5-44b4-91c7-8978e4b99288 admin admin] PATH_INFO: `/v3/projects/5d91488f572442e2b110634360929506/users/7f744064470147dfb6387999e8baf057/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:08.841431 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c6369-a9f5-44b4-91c7-8978e4b99288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d91488f572442e2b110634360929506, user_id=7f744064470147dfb6387999e8baf057, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:08.844121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c6369-a9f5-44b4-91c7-8978e4b99288 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:08.859877 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1252/2513] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:55:08 2022] PUT /identity/v3/projects/5d91488f572442e2b110634360929506/users/7f744064470147dfb6387999e8baf057/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:08.865679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f73a2-e6a6-4480-9071-282b782d5b19 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:08.865956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f73a2-e6a6-4480-9071-282b782d5b19 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:08.866225 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f73a2-e6a6-4480-9071-282b782d5b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:08.886429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-ab4f73a2-e6a6-4480-9071-282b782d5b19 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:08.887642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ab4f73a2-e6a6-4480-9071-282b782d5b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:08.963779 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1262/2514] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:55:08.974546 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c6369-a9f5-44b4-91c7-8978e4b99288 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:08.976375 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-078c6369-a9f5-44b4-91c7-8978e4b99288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:09.047871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c6369-a9f5-44b4-91c7-8978e4b99288 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:09.053687 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e423a77-f957-4338-96ba-d600ca8adb33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:09.054621 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e423a77-f957-4338-96ba-d600ca8adb33 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:09.055105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e423a77-f957-4338-96ba-d600ca8adb33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:09.055530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e423a77-f957-4338-96ba-d600ca8adb33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:09.072668 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e423a77-f957-4338-96ba-d600ca8adb33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:09.076966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e423a77-f957-4338-96ba-d600ca8adb33 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:09.098301 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1253/2515] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:55:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:55:10.043532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a8105-3c09-4076-b355-4cb4f6cf4982 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:10.044452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a8105-3c09-4076-b355-4cb4f6cf4982 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:10.044452 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a8105-3c09-4076-b355-4cb4f6cf4982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:10.075066 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-765a8105-3c09-4076-b355-4cb4f6cf4982 None None] MFA Rules not processed for user `7f744064470147dfb6387999e8baf057`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:10.076488 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-765a8105-3c09-4076-b355-4cb4f6cf4982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:10.111588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:10.153103 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1263/2516] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:10 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:55:10.157818 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e423a77-f957-4338-96ba-d600ca8adb33 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:10.168575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e423a77-f957-4338-96ba-d600ca8adb33 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:10.173996 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a02cb-8b81-4274-a3d9-36027396b781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:10.174677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a02cb-8b81-4274-a3d9-36027396b781 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:10.174974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a02cb-8b81-4274-a3d9-36027396b781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:10.175250 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a02cb-8b81-4274-a3d9-36027396b781 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:10.175897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a02cb-8b81-4274-a3d9-36027396b781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:10.177900 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a02cb-8b81-4274-a3d9-36027396b781 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:10.210168 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1254/2517] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:55:10 2022] POST /identity/v3/projects => generated 403 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:55:10.215026 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a8105-3c09-4076-b355-4cb4f6cf4982 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:10.227221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a8105-3c09-4076-b355-4cb4f6cf4982 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:10.299171 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4da1d2-062a-4e4b-8bf6-b6ae11e09efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:10.299924 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4da1d2-062a-4e4b-8bf6-b6ae11e09efe admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:10.300220 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4da1d2-062a-4e4b-8bf6-b6ae11e09efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:10.300342 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4da1d2-062a-4e4b-8bf6-b6ae11e09efe admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:10.300940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4da1d2-062a-4e4b-8bf6-b6ae11e09efe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:10.302868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4da1d2-062a-4e4b-8bf6-b6ae11e09efe admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:10.327234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1264/2518] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:55:10 2022] POST /identity/v3/users => generated 308 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:55:10.332142 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a02cb-8b81-4274-a3d9-36027396b781 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:10.344003 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a02cb-8b81-4274-a3d9-36027396b781 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:10.349527 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f228475-9bc1-4144-89f5-3c815f8d8bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:10.350149 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f228475-9bc1-4144-89f5-3c815f8d8bcb admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:10.350527 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f228475-9bc1-4144-89f5-3c815f8d8bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:10.350793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f228475-9bc1-4144-89f5-3c815f8d8bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:10.351589 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f228475-9bc1-4144-89f5-3c815f8d8bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:10.353385 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f228475-9bc1-4144-89f5-3c815f8d8bcb admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:10.365967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1255/2519] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:55:10.370580 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4da1d2-062a-4e4b-8bf6-b6ae11e09efe admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:10.381884 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4da1d2-062a-4e4b-8bf6-b6ae11e09efe admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:10.387238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3e7fb-b9d0-4218-939a-6d322947473a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:10.387912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3e7fb-b9d0-4218-939a-6d322947473a admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:10.388195 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3e7fb-b9d0-4218-939a-6d322947473a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:10.388468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3e7fb-b9d0-4218-939a-6d322947473a admin admin] PATH_INFO: `/v3/projects/9cff641d8ada4250b92350d84106c854/users/e10c5be2457e4252bf755f729687213e/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:10.405101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3e7fb-b9d0-4218-939a-6d322947473a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cff641d8ada4250b92350d84106c854, user_id=e10c5be2457e4252bf755f729687213e, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:10.407821 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3e7fb-b9d0-4218-939a-6d322947473a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:10.423258 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1265/2520] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:55:10 2022] PUT /identity/v3/projects/9cff641d8ada4250b92350d84106c854/users/e10c5be2457e4252bf755f729687213e/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:10.427514 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f228475-9bc1-4144-89f5-3c815f8d8bcb admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:10.434156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:10.438012 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f228475-9bc1-4144-89f5-3c815f8d8bcb admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:10.511959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746f7f4-13d9-4464-bed4-893e668e4ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:10.512447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746f7f4-13d9-4464-bed4-893e668e4ffc admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:10.512746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746f7f4-13d9-4464-bed4-893e668e4ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:10.513016 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746f7f4-13d9-4464-bed4-893e668e4ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:10.513691 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746f7f4-13d9-4464-bed4-893e668e4ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:10.515303 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746f7f4-13d9-4464-bed4-893e668e4ffc admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:10.526250 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1256/2521] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:55:10.530695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3e7fb-b9d0-4218-939a-6d322947473a admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:10.542270 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3e7fb-b9d0-4218-939a-6d322947473a admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:10.548705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f49cae-bbe0-4ccb-a2b9-52e82ef813bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:10.549460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f49cae-bbe0-4ccb-a2b9-52e82ef813bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:10.549738 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f49cae-bbe0-4ccb-a2b9-52e82ef813bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:10.550009 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f49cae-bbe0-4ccb-a2b9-52e82ef813bc admin admin] PATH_INFO: `/v3/projects/9cff641d8ada4250b92350d84106c854/users/e10c5be2457e4252bf755f729687213e/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:10.552180 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f49cae-bbe0-4ccb-a2b9-52e82ef813bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cff641d8ada4250b92350d84106c854, user_id=e10c5be2457e4252bf755f729687213e, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:10.554997 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f49cae-bbe0-4ccb-a2b9-52e82ef813bc admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:10.567054 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1266/2522] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:55:10 2022] PUT /identity/v3/projects/9cff641d8ada4250b92350d84106c854/users/e10c5be2457e4252bf755f729687213e/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:11.238346 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a6927-e3d0-4146-8cbc-ff357a06279b None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:11.238882 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a6927-e3d0-4146-8cbc-ff357a06279b None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:11.238882 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a6927-e3d0-4146-8cbc-ff357a06279b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:11.268380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-569a6927-e3d0-4146-8cbc-ff357a06279b None None] MFA Rules not processed for user `e10c5be2457e4252bf755f729687213e`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:11.269653 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-569a6927-e3d0-4146-8cbc-ff357a06279b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:11.341389 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1257/2523] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:11 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:55:12.424535 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f49cae-bbe0-4ccb-a2b9-52e82ef813bc admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:12.496450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f49cae-bbe0-4ccb-a2b9-52e82ef813bc admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:12.501699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dada4-7996-4c70-8d5e-3beb0b798d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:12.502338 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dada4-7996-4c70-8d5e-3beb0b798d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:12.502617 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dada4-7996-4c70-8d5e-3beb0b798d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:12.502889 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dada4-7996-4c70-8d5e-3beb0b798d44 admin admin] PATH_INFO: `/v3/users/7f744064470147dfb6387999e8baf057` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:12.504142 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862dada4-7996-4c70-8d5e-3beb0b798d44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f744064470147dfb6387999e8baf057)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:12.506631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862dada4-7996-4c70-8d5e-3beb0b798d44 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:12.532434 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:12.574024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-862dada4-7996-4c70-8d5e-3beb0b798d44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f744064470147dfb6387999e8baf057'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:12.582772 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-862dada4-7996-4c70-8d5e-3beb0b798d44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f744064470147dfb6387999e8baf057'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:12.583570 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-862dada4-7996-4c70-8d5e-3beb0b798d44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f744064470147dfb6387999e8baf057'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:12.584299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-862dada4-7996-4c70-8d5e-3beb0b798d44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f744064470147dfb6387999e8baf057'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:12.594179 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-862dada4-7996-4c70-8d5e-3beb0b798d44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f744064470147dfb6387999e8baf057'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:12.611904 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1267/2524] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:55:12 2022] DELETE /identity/v3/users/7f744064470147dfb6387999e8baf057 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:12.814018 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a6927-e3d0-4146-8cbc-ff357a06279b None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:12.832639 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a6927-e3d0-4146-8cbc-ff357a06279b None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:12.898316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:12.899136 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:12.899911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:12.900222 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7 admin admin] PATH_INFO: `/v3/projects/5d91488f572442e2b110634360929506` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:12.901528 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d91488f572442e2b110634360929506)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:12.903459 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:12.937182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:12.953601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d91488f572442e2b110634360929506', 'request_id': 'req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:12.960139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d91488f572442e2b110634360929506', 'request_id': 'req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:12.969272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d91488f572442e2b110634360929506', 'request_id': 'req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:12.980849 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7 admin admin] The token cache is being invalidate because project 5d91488f572442e2b110634360929506 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:55:12.981150 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:12.981768 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:12.983252 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1258/2525] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:55:12 2022] DELETE /identity/v3/projects/5d91488f572442e2b110634360929506 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:12.987608 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dada4-7996-4c70-8d5e-3beb0b798d44 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:12.988998 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-862dada4-7996-4c70-8d5e-3beb0b798d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:13.002565 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dada4-7996-4c70-8d5e-3beb0b798d44 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:13.071056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8bc50c-c238-4ea1-8441-74ae45aab9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:13.071799 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bc50c-c238-4ea1-8441-74ae45aab9e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:13.072127 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bc50c-c238-4ea1-8441-74ae45aab9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:13.072436 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bc50c-c238-4ea1-8441-74ae45aab9e4 admin admin] PATH_INFO: `/v3/users/e10c5be2457e4252bf755f729687213e` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:13.074201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8bc50c-c238-4ea1-8441-74ae45aab9e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e10c5be2457e4252bf755f729687213e)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:13.076848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8bc50c-c238-4ea1-8441-74ae45aab9e4 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:13.130609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0d8bc50c-c238-4ea1-8441-74ae45aab9e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e10c5be2457e4252bf755f729687213e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:13.137800 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0d8bc50c-c238-4ea1-8441-74ae45aab9e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e10c5be2457e4252bf755f729687213e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:13.138436 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0d8bc50c-c238-4ea1-8441-74ae45aab9e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e10c5be2457e4252bf755f729687213e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:13.139021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0d8bc50c-c238-4ea1-8441-74ae45aab9e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e10c5be2457e4252bf755f729687213e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:13.146167 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0d8bc50c-c238-4ea1-8441-74ae45aab9e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e10c5be2457e4252bf755f729687213e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:13.166924 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1268/2526] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:55:12 2022] DELETE /identity/v3/users/e10c5be2457e4252bf755f729687213e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:13.364272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:13.365909 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:13.379790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cddd5ed-3e7c-4e40-9934-2e10c4b695e7 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:13.438296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fdf47-bd9d-4392-b7d9-6b51fd4c2d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:13.438937 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fdf47-bd9d-4392-b7d9-6b51fd4c2d36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:13.439214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fdf47-bd9d-4392-b7d9-6b51fd4c2d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:13.439488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fdf47-bd9d-4392-b7d9-6b51fd4c2d36 admin admin] PATH_INFO: `/v3/projects/9cff641d8ada4250b92350d84106c854` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:13.440626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888fdf47-bd9d-4392-b7d9-6b51fd4c2d36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cff641d8ada4250b92350d84106c854)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:13.442614 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888fdf47-bd9d-4392-b7d9-6b51fd4c2d36 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:13.495621 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-888fdf47-bd9d-4392-b7d9-6b51fd4c2d36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cff641d8ada4250b92350d84106c854', 'request_id': 'req-888fdf47-bd9d-4392-b7d9-6b51fd4c2d36'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:13.501857 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-888fdf47-bd9d-4392-b7d9-6b51fd4c2d36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cff641d8ada4250b92350d84106c854', 'request_id': 'req-888fdf47-bd9d-4392-b7d9-6b51fd4c2d36'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:13.511970 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-888fdf47-bd9d-4392-b7d9-6b51fd4c2d36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cff641d8ada4250b92350d84106c854', 'request_id': 'req-888fdf47-bd9d-4392-b7d9-6b51fd4c2d36'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:13.522857 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-888fdf47-bd9d-4392-b7d9-6b51fd4c2d36 admin admin] The token cache is being invalidate because project 9cff641d8ada4250b92350d84106c854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:55:13.523147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-888fdf47-bd9d-4392-b7d9-6b51fd4c2d36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:13.523797 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-888fdf47-bd9d-4392-b7d9-6b51fd4c2d36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:13.525443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1259/2527] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:55:13 2022] DELETE /identity/v3/projects/9cff641d8ada4250b92350d84106c854 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:13.530553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e3c9e-8d05-4a9e-99b8-b115c5ff876c None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:13.531040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e3c9e-8d05-4a9e-99b8-b115c5ff876c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:13.531234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e3c9e-8d05-4a9e-99b8-b115c5ff876c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:13.551944 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-197e3c9e-8d05-4a9e-99b8-b115c5ff876c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:13.553177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-197e3c9e-8d05-4a9e-99b8-b115c5ff876c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:13.568243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:13.619928 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1269/2528] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:55:13.628245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672cd0a-3b4b-4681-8d61-58e3d572e5ea None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:13.628551 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672cd0a-3b4b-4681-8d61-58e3d572e5ea None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:13.628825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672cd0a-3b4b-4681-8d61-58e3d572e5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:13.649789 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-7672cd0a-3b4b-4681-8d61-58e3d572e5ea None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:13.651021 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7672cd0a-3b4b-4681-8d61-58e3d572e5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:13.660534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1260/2529] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:55:13.665126 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e3c9e-8d05-4a9e-99b8-b115c5ff876c None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:13.675546 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e3c9e-8d05-4a9e-99b8-b115c5ff876c None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:13.681302 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b9bb1-1952-41a2-987e-82eaba7da9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:13.681912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b9bb1-1952-41a2-987e-82eaba7da9fc admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:13.682192 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b9bb1-1952-41a2-987e-82eaba7da9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:13.682528 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b9bb1-1952-41a2-987e-82eaba7da9fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:13.683178 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b9bb1-1952-41a2-987e-82eaba7da9fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:55:13.683482 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b9bb1-1952-41a2-987e-82eaba7da9fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:13.685118 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b9bb1-1952-41a2-987e-82eaba7da9fc admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:13.698034 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1270/2530] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:55:13.702939 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672cd0a-3b4b-4681-8d61-58e3d572e5ea None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:13.712678 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672cd0a-3b4b-4681-8d61-58e3d572e5ea None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:13.717883 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb5ef8-69df-486d-8662-d6bdd7c838ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:13.718489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb5ef8-69df-486d-8662-d6bdd7c838ef admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:13.718763 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb5ef8-69df-486d-8662-d6bdd7c838ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:13.719027 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb5ef8-69df-486d-8662-d6bdd7c838ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:13.719745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb5ef8-69df-486d-8662-d6bdd7c838ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:13.721753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb5ef8-69df-486d-8662-d6bdd7c838ef admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:13.756096 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1261/2531] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:55:13 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:55:13.761101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b9bb1-1952-41a2-987e-82eaba7da9fc admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:13.775553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b9bb1-1952-41a2-987e-82eaba7da9fc admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:13.843654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585cf2f-cc55-49bf-8632-ced4d523c462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:13.844514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585cf2f-cc55-49bf-8632-ced4d523c462 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:13.844800 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585cf2f-cc55-49bf-8632-ced4d523c462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:13.845070 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585cf2f-cc55-49bf-8632-ced4d523c462 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:13.845916 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585cf2f-cc55-49bf-8632-ced4d523c462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:13.848644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585cf2f-cc55-49bf-8632-ced4d523c462 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:13.877162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1271/2532] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:55:13 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:55:13.881820 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb5ef8-69df-486d-8662-d6bdd7c838ef admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:13.893736 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb5ef8-69df-486d-8662-d6bdd7c838ef admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:13.899443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad1114-f601-4913-87fb-417d53ecd570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:13.900113 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad1114-f601-4913-87fb-417d53ecd570 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:13.900416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad1114-f601-4913-87fb-417d53ecd570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:13.900685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad1114-f601-4913-87fb-417d53ecd570 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:13.901556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad1114-f601-4913-87fb-417d53ecd570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:13.903182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad1114-f601-4913-87fb-417d53ecd570 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:13.915306 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1262/2533] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:55:13.920143 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585cf2f-cc55-49bf-8632-ced4d523c462 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:13.931401 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585cf2f-cc55-49bf-8632-ced4d523c462 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:13.937299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690fbebd-fe2d-4733-b648-01f3002db0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:13.937914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fbebd-fe2d-4733-b648-01f3002db0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:13.938237 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fbebd-fe2d-4733-b648-01f3002db0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:13.938539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fbebd-fe2d-4733-b648-01f3002db0f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:13.939240 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690fbebd-fe2d-4733-b648-01f3002db0f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:13.940933 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690fbebd-fe2d-4733-b648-01f3002db0f5 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:13.953037 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1272/2534] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:55:13.958195 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad1114-f601-4913-87fb-417d53ecd570 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:13.969035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad1114-f601-4913-87fb-417d53ecd570 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:13.973847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff3b19-05af-4e7b-91cb-db5c2afa06fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:13.974516 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff3b19-05af-4e7b-91cb-db5c2afa06fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:13.974795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff3b19-05af-4e7b-91cb-db5c2afa06fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:13.975071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff3b19-05af-4e7b-91cb-db5c2afa06fa admin admin] PATH_INFO: `/v3/projects/7e939454dcc54b00b19da6f40001272e/users/6b6d270ff47448f98b351924217e8d73/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:13.977606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:13.993793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff3b19-05af-4e7b-91cb-db5c2afa06fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e939454dcc54b00b19da6f40001272e, user_id=6b6d270ff47448f98b351924217e8d73, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:13.996450 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff3b19-05af-4e7b-91cb-db5c2afa06fa admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:14.010227 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1263/2535] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:55:13 2022] PUT /identity/v3/projects/7e939454dcc54b00b19da6f40001272e/users/6b6d270ff47448f98b351924217e8d73/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:14.022649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e99b3-f5e9-4c9d-a654-e6191fcf7544 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:14.022910 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e99b3-f5e9-4c9d-a654-e6191fcf7544 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:14.023332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e99b3-f5e9-4c9d-a654-e6191fcf7544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:14.059450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-0a7e99b3-f5e9-4c9d-a654-e6191fcf7544 None None] MFA Rules not processed for user `6b6d270ff47448f98b351924217e8d73`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:14.060717 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-0a7e99b3-f5e9-4c9d-a654-e6191fcf7544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:14.137243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1273/2536] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:55:14.143670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff3b19-05af-4e7b-91cb-db5c2afa06fa admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:14.211704 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff3b19-05af-4e7b-91cb-db5c2afa06fa admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:14.217156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2f6c6-a439-44c4-a102-6d00dd9e7ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:14.217846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2f6c6-a439-44c4-a102-6d00dd9e7ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:14.218122 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2f6c6-a439-44c4-a102-6d00dd9e7ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:14.218391 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2f6c6-a439-44c4-a102-6d00dd9e7ab4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:14.219035 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2f6c6-a439-44c4-a102-6d00dd9e7ab4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:14.220978 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2f6c6-a439-44c4-a102-6d00dd9e7ab4 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:14.259368 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1264/2537] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:55:14 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:55:14.264054 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e99b3-f5e9-4c9d-a654-e6191fcf7544 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:14.276138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e99b3-f5e9-4c9d-a654-e6191fcf7544 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:14.292816 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:14.344289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae16949-35fb-47df-b892-3d5c372379cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:14.345438 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae16949-35fb-47df-b892-3d5c372379cb admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:14.345721 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae16949-35fb-47df-b892-3d5c372379cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:14.346509 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae16949-35fb-47df-b892-3d5c372379cb admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:14.347339 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae16949-35fb-47df-b892-3d5c372379cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:14.349949 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae16949-35fb-47df-b892-3d5c372379cb admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:14.373836 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1274/2538] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:55:14 2022] POST /identity/v3/users => generated 296 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:55:14.378025 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2f6c6-a439-44c4-a102-6d00dd9e7ab4 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:14.388801 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2f6c6-a439-44c4-a102-6d00dd9e7ab4 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:14.393967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ff80c-bf6a-4c6b-a55b-31648060d822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:14.394554 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ff80c-bf6a-4c6b-a55b-31648060d822 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:14.394854 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ff80c-bf6a-4c6b-a55b-31648060d822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:14.395120 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ff80c-bf6a-4c6b-a55b-31648060d822 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:14.395762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ff80c-bf6a-4c6b-a55b-31648060d822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:14.397468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ff80c-bf6a-4c6b-a55b-31648060d822 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:14.409677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1265/2539] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:55:14.414198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae16949-35fb-47df-b892-3d5c372379cb admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:14.425654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae16949-35fb-47df-b892-3d5c372379cb admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:14.434026 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070917c1-3bfb-4910-93c2-de1d76cf5a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:14.435175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070917c1-3bfb-4910-93c2-de1d76cf5a32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:14.435654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070917c1-3bfb-4910-93c2-de1d76cf5a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:14.436114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070917c1-3bfb-4910-93c2-de1d76cf5a32 admin admin] PATH_INFO: `/v3/projects/ff9188afbea24187bd7b23bbc0ab0cd2/users/3e3e194e000a4ae184c6067593f66dd4/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:14.453991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070917c1-3bfb-4910-93c2-de1d76cf5a32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff9188afbea24187bd7b23bbc0ab0cd2, user_id=3e3e194e000a4ae184c6067593f66dd4, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:14.457232 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070917c1-3bfb-4910-93c2-de1d76cf5a32 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:14.472532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1275/2540] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:55:14 2022] PUT /identity/v3/projects/ff9188afbea24187bd7b23bbc0ab0cd2/users/3e3e194e000a4ae184c6067593f66dd4/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:14.477053 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ff80c-bf6a-4c6b-a55b-31648060d822 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:14.488985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ff80c-bf6a-4c6b-a55b-31648060d822 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:14.550137 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce590375-0e80-4afd-964f-c1f6e007672e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:14.550756 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce590375-0e80-4afd-964f-c1f6e007672e admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:14.551116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce590375-0e80-4afd-964f-c1f6e007672e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:14.551462 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce590375-0e80-4afd-964f-c1f6e007672e admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:14.552110 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce590375-0e80-4afd-964f-c1f6e007672e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:14.553783 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce590375-0e80-4afd-964f-c1f6e007672e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:14.564394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1266/2541] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:55:14.568891 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070917c1-3bfb-4910-93c2-de1d76cf5a32 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:14.579870 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070917c1-3bfb-4910-93c2-de1d76cf5a32 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:14.586504 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57e3a7-914c-4f2e-948e-d72983543f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:14.587182 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57e3a7-914c-4f2e-948e-d72983543f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:14.587461 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57e3a7-914c-4f2e-948e-d72983543f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:14.587711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57e3a7-914c-4f2e-948e-d72983543f71 admin admin] PATH_INFO: `/v3/projects/ff9188afbea24187bd7b23bbc0ab0cd2/users/3e3e194e000a4ae184c6067593f66dd4/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:14.589893 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57e3a7-914c-4f2e-948e-d72983543f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff9188afbea24187bd7b23bbc0ab0cd2, user_id=3e3e194e000a4ae184c6067593f66dd4, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:14.592858 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57e3a7-914c-4f2e-948e-d72983543f71 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:14.605663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1276/2542] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:55:14 2022] PUT /identity/v3/projects/ff9188afbea24187bd7b23bbc0ab0cd2/users/3e3e194e000a4ae184c6067593f66dd4/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:55:14.616040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863f79d-6697-4b9d-8f48-500d373cac08 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:14.616344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863f79d-6697-4b9d-8f48-500d373cac08 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:14.616618 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863f79d-6697-4b9d-8f48-500d373cac08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:14.647572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f863f79d-6697-4b9d-8f48-500d373cac08 None None] MFA Rules not processed for user `3e3e194e000a4ae184c6067593f66dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:14.648825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f863f79d-6697-4b9d-8f48-500d373cac08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:14.700509 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:14.737148 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1267/2543] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:14 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:55:14.749431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57e3a7-914c-4f2e-948e-d72983543f71 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:14.751278 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-8a57e3a7-914c-4f2e-948e-d72983543f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:14.857602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57e3a7-914c-4f2e-948e-d72983543f71 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:14.863699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812ac2d-c276-4705-bc74-d2e78cc7d11d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:14.864698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812ac2d-c276-4705-bc74-d2e78cc7d11d service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:14.865295 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812ac2d-c276-4705-bc74-d2e78cc7d11d service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:14.865760 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812ac2d-c276-4705-bc74-d2e78cc7d11d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:14.882675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812ac2d-c276-4705-bc74-d2e78cc7d11d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:14.885540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812ac2d-c276-4705-bc74-d2e78cc7d11d service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:14.904187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1277/2544] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:55:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:55:15.020277 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863f79d-6697-4b9d-8f48-500d373cac08 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:15.032048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863f79d-6697-4b9d-8f48-500d373cac08 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:15.036844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5f0eb-8aa7-4b30-9b5c-21c025159d10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:15.037530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5f0eb-8aa7-4b30-9b5c-21c025159d10 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:15.037817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5f0eb-8aa7-4b30-9b5c-21c025159d10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:15.038109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5f0eb-8aa7-4b30-9b5c-21c025159d10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:15.110718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5f0eb-8aa7-4b30-9b5c-21c025159d10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:15.112625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5f0eb-8aa7-4b30-9b5c-21c025159d10 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:15.124639 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1268/2545] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:55:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:55:24.895989 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=63,keystone:INSERT=5 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:25.118829 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=38 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:26.355460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812ac2d-c276-4705-bc74-d2e78cc7d11d service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:26.426504 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812ac2d-c276-4705-bc74-d2e78cc7d11d service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:26.431353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e242f2f6-be11-486b-99e1-448bcecd8ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:26.432049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242f2f6-be11-486b-99e1-448bcecd8ae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:26.432325 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242f2f6-be11-486b-99e1-448bcecd8ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:26.432728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242f2f6-be11-486b-99e1-448bcecd8ae6 admin admin] PATH_INFO: `/v3/users/6b6d270ff47448f98b351924217e8d73` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:26.433957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e242f2f6-be11-486b-99e1-448bcecd8ae6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b6d270ff47448f98b351924217e8d73)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:26.435923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e242f2f6-be11-486b-99e1-448bcecd8ae6 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:26.497398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-e242f2f6-be11-486b-99e1-448bcecd8ae6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b6d270ff47448f98b351924217e8d73'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:26.508003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-e242f2f6-be11-486b-99e1-448bcecd8ae6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b6d270ff47448f98b351924217e8d73'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:26.508863 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-e242f2f6-be11-486b-99e1-448bcecd8ae6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b6d270ff47448f98b351924217e8d73'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:26.509737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-e242f2f6-be11-486b-99e1-448bcecd8ae6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b6d270ff47448f98b351924217e8d73'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:26.519849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-e242f2f6-be11-486b-99e1-448bcecd8ae6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b6d270ff47448f98b351924217e8d73'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:26.540121 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1278/2546] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:55:26 2022] DELETE /identity/v3/users/6b6d270ff47448f98b351924217e8d73 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:26.546921 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c152c3-196e-4780-be54-d4fe4f80117d None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:26.547947 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c152c3-196e-4780-be54-d4fe4f80117d None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:26.547947 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c152c3-196e-4780-be54-d4fe4f80117d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:26.572466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-46c152c3-196e-4780-be54-d4fe4f80117d None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:26.573813 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-46c152c3-196e-4780-be54-d4fe4f80117d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:26.640192 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1269/2547] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:55:26.651170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e242f2f6-be11-486b-99e1-448bcecd8ae6 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:26.652970 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-e242f2f6-be11-486b-99e1-448bcecd8ae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:26.842036 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e242f2f6-be11-486b-99e1-448bcecd8ae6 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:26.846454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f274c4-87ab-4995-be7d-1cf9cfc859f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:26.847083 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f274c4-87ab-4995-be7d-1cf9cfc859f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:26.847361 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f274c4-87ab-4995-be7d-1cf9cfc859f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:26.847625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f274c4-87ab-4995-be7d-1cf9cfc859f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:26.858496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f274c4-87ab-4995-be7d-1cf9cfc859f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:26.859944 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f274c4-87ab-4995-be7d-1cf9cfc859f6 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:26.871631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1279/2548] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:55:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:55:27.167886 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c152c3-196e-4780-be54-d4fe4f80117d None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:27.170094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-46c152c3-196e-4780-be54-d4fe4f80117d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:27.186692 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c152c3-196e-4780-be54-d4fe4f80117d None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:27.196015 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec915654-e2b8-4fa7-9532-3779066ee16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:27.196673 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec915654-e2b8-4fa7-9532-3779066ee16c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:27.196965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec915654-e2b8-4fa7-9532-3779066ee16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:27.197278 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec915654-e2b8-4fa7-9532-3779066ee16c admin admin] PATH_INFO: `/v3/projects/7e939454dcc54b00b19da6f40001272e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:27.198779 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec915654-e2b8-4fa7-9532-3779066ee16c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e939454dcc54b00b19da6f40001272e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:27.201432 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec915654-e2b8-4fa7-9532-3779066ee16c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:27.260414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ec915654-e2b8-4fa7-9532-3779066ee16c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e939454dcc54b00b19da6f40001272e', 'request_id': 'req-ec915654-e2b8-4fa7-9532-3779066ee16c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:27.267745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ec915654-e2b8-4fa7-9532-3779066ee16c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e939454dcc54b00b19da6f40001272e', 'request_id': 'req-ec915654-e2b8-4fa7-9532-3779066ee16c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:27.279171 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ec915654-e2b8-4fa7-9532-3779066ee16c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e939454dcc54b00b19da6f40001272e', 'request_id': 'req-ec915654-e2b8-4fa7-9532-3779066ee16c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:27.294570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ec915654-e2b8-4fa7-9532-3779066ee16c admin admin] The token cache is being invalidate because project 7e939454dcc54b00b19da6f40001272e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:55:27.294570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ec915654-e2b8-4fa7-9532-3779066ee16c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:27.294570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ec915654-e2b8-4fa7-9532-3779066ee16c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:27.295390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1270/2549] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:55:27 2022] DELETE /identity/v3/projects/7e939454dcc54b00b19da6f40001272e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:27.303981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f274c4-87ab-4995-be7d-1cf9cfc859f6 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:27.316559 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f274c4-87ab-4995-be7d-1cf9cfc859f6 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:27.340559 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:27.388318 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170a4c4-ff71-4270-bc89-f490eb2f2897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:27.389772 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170a4c4-ff71-4270-bc89-f490eb2f2897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:27.390210 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170a4c4-ff71-4270-bc89-f490eb2f2897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:27.390665 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170a4c4-ff71-4270-bc89-f490eb2f2897 admin admin] PATH_INFO: `/v3/users/3e3e194e000a4ae184c6067593f66dd4` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:27.392032 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170a4c4-ff71-4270-bc89-f490eb2f2897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e3e194e000a4ae184c6067593f66dd4)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:27.394159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170a4c4-ff71-4270-bc89-f490eb2f2897 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:27.450055 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c170a4c4-ff71-4270-bc89-f490eb2f2897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e3e194e000a4ae184c6067593f66dd4'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:27.459365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c170a4c4-ff71-4270-bc89-f490eb2f2897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e3e194e000a4ae184c6067593f66dd4'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:27.460127 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c170a4c4-ff71-4270-bc89-f490eb2f2897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e3e194e000a4ae184c6067593f66dd4'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:27.460796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c170a4c4-ff71-4270-bc89-f490eb2f2897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e3e194e000a4ae184c6067593f66dd4'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:27.468965 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c170a4c4-ff71-4270-bc89-f490eb2f2897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e3e194e000a4ae184c6067593f66dd4'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:27.488142 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1280/2550] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:55:27 2022] DELETE /identity/v3/users/3e3e194e000a4ae184c6067593f66dd4 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:27.826534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec915654-e2b8-4fa7-9532-3779066ee16c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:27.828729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ec915654-e2b8-4fa7-9532-3779066ee16c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:27.845084 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec915654-e2b8-4fa7-9532-3779066ee16c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:27.917259 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef668a4-ef59-4ee4-964f-f03066bd4a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:27.917921 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef668a4-ef59-4ee4-964f-f03066bd4a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:27.918197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef668a4-ef59-4ee4-964f-f03066bd4a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:27.918467 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef668a4-ef59-4ee4-964f-f03066bd4a3f admin admin] PATH_INFO: `/v3/projects/ff9188afbea24187bd7b23bbc0ab0cd2` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:27.919659 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef668a4-ef59-4ee4-964f-f03066bd4a3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff9188afbea24187bd7b23bbc0ab0cd2)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:27.921638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef668a4-ef59-4ee4-964f-f03066bd4a3f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:27.957731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:27.980172 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-1ef668a4-ef59-4ee4-964f-f03066bd4a3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff9188afbea24187bd7b23bbc0ab0cd2', 'request_id': 'req-1ef668a4-ef59-4ee4-964f-f03066bd4a3f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:27.987354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-1ef668a4-ef59-4ee4-964f-f03066bd4a3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff9188afbea24187bd7b23bbc0ab0cd2', 'request_id': 'req-1ef668a4-ef59-4ee4-964f-f03066bd4a3f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:27.998030 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-1ef668a4-ef59-4ee4-964f-f03066bd4a3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff9188afbea24187bd7b23bbc0ab0cd2', 'request_id': 'req-1ef668a4-ef59-4ee4-964f-f03066bd4a3f'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:28.009888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-1ef668a4-ef59-4ee4-964f-f03066bd4a3f admin admin] The token cache is being invalidate because project ff9188afbea24187bd7b23bbc0ab0cd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:55:28.010182 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-1ef668a4-ef59-4ee4-964f-f03066bd4a3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:28.010901 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-1ef668a4-ef59-4ee4-964f-f03066bd4a3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:28.012419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1271/2551] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:55:27 2022] DELETE /identity/v3/projects/ff9188afbea24187bd7b23bbc0ab0cd2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:28.017564 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fe67f-33d2-4159-9192-56e3136d7284 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:28.017854 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fe67f-33d2-4159-9192-56e3136d7284 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:28.018089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fe67f-33d2-4159-9192-56e3136d7284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:28.039640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-cb7fe67f-33d2-4159-9192-56e3136d7284 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:28.040925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-cb7fe67f-33d2-4159-9192-56e3136d7284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:28.106306 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1281/2552] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:55:28.115582 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb5f3-a3f4-4058-be77-e113844738de None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:28.115866 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb5f3-a3f4-4058-be77-e113844738de None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:28.116143 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb5f3-a3f4-4058-be77-e113844738de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:28.138398 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-0e5bb5f3-a3f4-4058-be77-e113844738de None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:28.139677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0e5bb5f3-a3f4-4058-be77-e113844738de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:28.148127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1272/2553] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:55:28.152719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fe67f-33d2-4159-9192-56e3136d7284 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:28.163062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fe67f-33d2-4159-9192-56e3136d7284 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:28.168577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad2133-8af5-40b6-b471-674711dc99bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:28.169378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad2133-8af5-40b6-b471-674711dc99bc admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:28.169658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad2133-8af5-40b6-b471-674711dc99bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:28.169931 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad2133-8af5-40b6-b471-674711dc99bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:28.170671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad2133-8af5-40b6-b471-674711dc99bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:55:28.170972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad2133-8af5-40b6-b471-674711dc99bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:28.172642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad2133-8af5-40b6-b471-674711dc99bc admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:28.185574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1282/2554] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:55:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:55:28.190596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bb5f3-a3f4-4058-be77-e113844738de None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:28.203798 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bb5f3-a3f4-4058-be77-e113844738de None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:28.212457 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d070e5e-a554-4699-9f56-87a1185e411c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:28.213539 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d070e5e-a554-4699-9f56-87a1185e411c admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:28.214050 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d070e5e-a554-4699-9f56-87a1185e411c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:28.214529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d070e5e-a554-4699-9f56-87a1185e411c admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:28.215673 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d070e5e-a554-4699-9f56-87a1185e411c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:28.218679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d070e5e-a554-4699-9f56-87a1185e411c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:28.256252 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1273/2555] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:55:28 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:55:28.261487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad2133-8af5-40b6-b471-674711dc99bc admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:28.274208 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad2133-8af5-40b6-b471-674711dc99bc admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:28.304968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:28.342511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19cf2c7-a04f-4f83-b9dd-b73c8c7e9842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:28.343162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19cf2c7-a04f-4f83-b9dd-b73c8c7e9842 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:28.343437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19cf2c7-a04f-4f83-b9dd-b73c8c7e9842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:28.343702 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19cf2c7-a04f-4f83-b9dd-b73c8c7e9842 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:28.344366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19cf2c7-a04f-4f83-b9dd-b73c8c7e9842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:28.346347 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19cf2c7-a04f-4f83-b9dd-b73c8c7e9842 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:28.372635 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1283/2556] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:55:28 2022] POST /identity/v3/users => generated 283 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:55:28.377566 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d070e5e-a554-4699-9f56-87a1185e411c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:28.389266 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d070e5e-a554-4699-9f56-87a1185e411c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:28.394855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3be220-8a9f-472b-ac29-06221ac71124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:28.395495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3be220-8a9f-472b-ac29-06221ac71124 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:28.395768 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3be220-8a9f-472b-ac29-06221ac71124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:28.396040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3be220-8a9f-472b-ac29-06221ac71124 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:28.396719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3be220-8a9f-472b-ac29-06221ac71124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:28.398446 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3be220-8a9f-472b-ac29-06221ac71124 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:28.410627 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1274/2557] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:55:28.415266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19cf2c7-a04f-4f83-b9dd-b73c8c7e9842 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:28.426562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19cf2c7-a04f-4f83-b9dd-b73c8c7e9842 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:28.432703 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db05dc-4e0a-4e66-94d2-9298ff68110a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:28.433410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db05dc-4e0a-4e66-94d2-9298ff68110a admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:28.433688 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db05dc-4e0a-4e66-94d2-9298ff68110a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:28.433946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db05dc-4e0a-4e66-94d2-9298ff68110a admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:28.434628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db05dc-4e0a-4e66-94d2-9298ff68110a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:28.436307 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db05dc-4e0a-4e66-94d2-9298ff68110a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:28.447836 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1284/2558] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:55:28.455906 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3be220-8a9f-472b-ac29-06221ac71124 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:28.471736 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3be220-8a9f-472b-ac29-06221ac71124 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:28.477166 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f02059-c198-42d1-b71a-74f53796c473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:28.477849 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f02059-c198-42d1-b71a-74f53796c473 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:28.478124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f02059-c198-42d1-b71a-74f53796c473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:28.478392 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f02059-c198-42d1-b71a-74f53796c473 admin admin] PATH_INFO: `/v3/projects/8abfa26056a84b49a9a52230e81f8a1d/users/02ec50710561465ab7726d220338a2b1/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:28.495396 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f02059-c198-42d1-b71a-74f53796c473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8abfa26056a84b49a9a52230e81f8a1d, user_id=02ec50710561465ab7726d220338a2b1, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:28.498142 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f02059-c198-42d1-b71a-74f53796c473 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:28.514644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1275/2559] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:55:28 2022] PUT /identity/v3/projects/8abfa26056a84b49a9a52230e81f8a1d/users/02ec50710561465ab7726d220338a2b1/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:28.527630 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92968d54-55a1-4df6-9378-fc6a90c21a28 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:28.527963 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92968d54-55a1-4df6-9378-fc6a90c21a28 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:28.528176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92968d54-55a1-4df6-9378-fc6a90c21a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:28.560178 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-92968d54-55a1-4df6-9378-fc6a90c21a28 None None] MFA Rules not processed for user `02ec50710561465ab7726d220338a2b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:28.561557 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-92968d54-55a1-4df6-9378-fc6a90c21a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:28.644605 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1285/2560] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:55:28.649368 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f02059-c198-42d1-b71a-74f53796c473 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:28.719317 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f02059-c198-42d1-b71a-74f53796c473 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:28.724812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad9beb-1a52-4334-b843-82f2aacac92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:28.725473 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad9beb-1a52-4334-b843-82f2aacac92f admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:28.725752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad9beb-1a52-4334-b843-82f2aacac92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:28.726023 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad9beb-1a52-4334-b843-82f2aacac92f admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:28.726669 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad9beb-1a52-4334-b843-82f2aacac92f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:28.728611 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad9beb-1a52-4334-b843-82f2aacac92f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:28.763892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1276/2561] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:55:28 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:55:28.765207 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92968d54-55a1-4df6-9378-fc6a90c21a28 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:28.777447 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92968d54-55a1-4df6-9378-fc6a90c21a28 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:28.842820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4d7aa-f488-40b1-9613-f4e1eb6d34a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:28.843448 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4d7aa-f488-40b1-9613-f4e1eb6d34a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:28.844616 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4d7aa-f488-40b1-9613-f4e1eb6d34a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:28.845640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4d7aa-f488-40b1-9613-f4e1eb6d34a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:28.846327 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4d7aa-f488-40b1-9613-f4e1eb6d34a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:28.848257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4d7aa-f488-40b1-9613-f4e1eb6d34a8 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:28.874290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1286/2562] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:55:28 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:55:28.880130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad9beb-1a52-4334-b843-82f2aacac92f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:28.896874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad9beb-1a52-4334-b843-82f2aacac92f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:28.905107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19e40c-b125-49af-9142-877098c915e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:28.906183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19e40c-b125-49af-9142-877098c915e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:28.906685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19e40c-b125-49af-9142-877098c915e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:28.907181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19e40c-b125-49af-9142-877098c915e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:28.908380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19e40c-b125-49af-9142-877098c915e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:28.912379 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19e40c-b125-49af-9142-877098c915e9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:28.932758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1277/2563] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:55:28.937673 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4d7aa-f488-40b1-9613-f4e1eb6d34a8 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:28.948931 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4d7aa-f488-40b1-9613-f4e1eb6d34a8 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:28.954228 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6dbe5-15ef-4fb2-b771-3d31eb5e6ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:28.954896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6dbe5-15ef-4fb2-b771-3d31eb5e6ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:28.955215 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6dbe5-15ef-4fb2-b771-3d31eb5e6ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:28.955477 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6dbe5-15ef-4fb2-b771-3d31eb5e6ee3 admin admin] PATH_INFO: `/v3/projects/b3fec316f89e4f6da60966e578e9b4d8/users/3ff14623626d48689d0003e1de51bbe7/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:28.972924 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6dbe5-15ef-4fb2-b771-3d31eb5e6ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3fec316f89e4f6da60966e578e9b4d8, user_id=3ff14623626d48689d0003e1de51bbe7, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:28.975636 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6dbe5-15ef-4fb2-b771-3d31eb5e6ee3 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:28.989663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1287/2564] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:55:28 2022] PUT /identity/v3/projects/b3fec316f89e4f6da60966e578e9b4d8/users/3ff14623626d48689d0003e1de51bbe7/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:28.993805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19e40c-b125-49af-9142-877098c915e9 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:29.004596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19e40c-b125-49af-9142-877098c915e9 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:29.027637 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:29.070774 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb573e-07bf-4623-a606-39ab88f93d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:29.071412 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb573e-07bf-4623-a606-39ab88f93d4e admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:29.071688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb573e-07bf-4623-a606-39ab88f93d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:29.071961 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb573e-07bf-4623-a606-39ab88f93d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:29.072629 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb573e-07bf-4623-a606-39ab88f93d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:29.074462 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb573e-07bf-4623-a606-39ab88f93d4e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:29.090661 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1278/2565] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:55:29.095934 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6dbe5-15ef-4fb2-b771-3d31eb5e6ee3 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:29.105331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:29.114178 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6dbe5-15ef-4fb2-b771-3d31eb5e6ee3 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:29.120171 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5181de-c6c9-48d5-a080-40c083450ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:29.121037 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5181de-c6c9-48d5-a080-40c083450ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:29.121407 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5181de-c6c9-48d5-a080-40c083450ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:29.121727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5181de-c6c9-48d5-a080-40c083450ce7 admin admin] PATH_INFO: `/v3/projects/b3fec316f89e4f6da60966e578e9b4d8/users/3ff14623626d48689d0003e1de51bbe7/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:29.124159 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5181de-c6c9-48d5-a080-40c083450ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3fec316f89e4f6da60966e578e9b4d8, user_id=3ff14623626d48689d0003e1de51bbe7, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:29.126987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5181de-c6c9-48d5-a080-40c083450ce7 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:29.139167 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1288/2566] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:55:29 2022] PUT /identity/v3/projects/b3fec316f89e4f6da60966e578e9b4d8/users/3ff14623626d48689d0003e1de51bbe7/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:29.150714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b1a59-6931-4632-88e5-a486cca32615 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:29.151014 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b1a59-6931-4632-88e5-a486cca32615 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:29.151500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b1a59-6931-4632-88e5-a486cca32615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:29.186816 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-e21b1a59-6931-4632-88e5-a486cca32615 None None] MFA Rules not processed for user `3ff14623626d48689d0003e1de51bbe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:29.188287 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e21b1a59-6931-4632-88e5-a486cca32615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:29.266245 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1279/2567] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:55:29.278922 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5181de-c6c9-48d5-a080-40c083450ce7 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:29.281271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ae5181de-c6c9-48d5-a080-40c083450ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:29.350076 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5181de-c6c9-48d5-a080-40c083450ce7 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:29.353994 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e917fc0-d020-469d-b26b-cdcc899aa838 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:29.354619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e917fc0-d020-469d-b26b-cdcc899aa838 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:29.354896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e917fc0-d020-469d-b26b-cdcc899aa838 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:29.355168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e917fc0-d020-469d-b26b-cdcc899aa838 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:29.365406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e917fc0-d020-469d-b26b-cdcc899aa838 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:29.366857 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e917fc0-d020-469d-b26b-cdcc899aa838 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:29.378039 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1289/2568] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:55:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:55:29.481902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b1a59-6931-4632-88e5-a486cca32615 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:29.502648 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b1a59-6931-4632-88e5-a486cca32615 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:29.507500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffa73e-a4d9-4e47-926d-c08ea6811b2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:29.508156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffa73e-a4d9-4e47-926d-c08ea6811b2d service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:29.508434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffa73e-a4d9-4e47-926d-c08ea6811b2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:29.508707 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffa73e-a4d9-4e47-926d-c08ea6811b2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:29.579655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffa73e-a4d9-4e47-926d-c08ea6811b2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:29.581577 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffa73e-a4d9-4e47-926d-c08ea6811b2d service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:29.592290 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1280/2569] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:55:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:55:39.373018 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:39.587434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=80 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:41.479395 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e917fc0-d020-469d-b26b-cdcc899aa838 service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:41.564808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e917fc0-d020-469d-b26b-cdcc899aa838 service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:41.570341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed63fcc-4c08-4063-a550-1afd6756559b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:41.571120 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed63fcc-4c08-4063-a550-1afd6756559b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:41.571396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed63fcc-4c08-4063-a550-1afd6756559b admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:41.571663 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed63fcc-4c08-4063-a550-1afd6756559b admin admin] PATH_INFO: `/v3/users/02ec50710561465ab7726d220338a2b1` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:41.572857 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed63fcc-4c08-4063-a550-1afd6756559b admin admin] RBAC: Authorizing `identity:delete_user(user_id=02ec50710561465ab7726d220338a2b1)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:41.574841 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed63fcc-4c08-4063-a550-1afd6756559b admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:41.632970 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-3ed63fcc-4c08-4063-a550-1afd6756559b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02ec50710561465ab7726d220338a2b1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:41.642353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-3ed63fcc-4c08-4063-a550-1afd6756559b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02ec50710561465ab7726d220338a2b1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:41.643136 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-3ed63fcc-4c08-4063-a550-1afd6756559b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02ec50710561465ab7726d220338a2b1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:41.643866 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-3ed63fcc-4c08-4063-a550-1afd6756559b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02ec50710561465ab7726d220338a2b1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:41.652243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-3ed63fcc-4c08-4063-a550-1afd6756559b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02ec50710561465ab7726d220338a2b1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:41.671062 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1290/2570] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:55:41 2022] DELETE /identity/v3/users/02ec50710561465ab7726d220338a2b1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:41.676275 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea646572-6e79-4624-b74f-802d74c5a305 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:41.676844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea646572-6e79-4624-b74f-802d74c5a305 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:41.676844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea646572-6e79-4624-b74f-802d74c5a305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:41.697916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ea646572-6e79-4624-b74f-802d74c5a305 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:41.699169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ea646572-6e79-4624-b74f-802d74c5a305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:41.768669 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1281/2571] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:55:41.781511 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed63fcc-4c08-4063-a550-1afd6756559b admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:41.783669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-3ed63fcc-4c08-4063-a550-1afd6756559b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:41.857111 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed63fcc-4c08-4063-a550-1afd6756559b admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:41.861194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cc61e-7809-4ea1-b26f-741acb563d44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:41.861855 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cc61e-7809-4ea1-b26f-741acb563d44 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:41.862133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cc61e-7809-4ea1-b26f-741acb563d44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:41.862405 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cc61e-7809-4ea1-b26f-741acb563d44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:41.873057 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cc61e-7809-4ea1-b26f-741acb563d44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:41.874600 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cc61e-7809-4ea1-b26f-741acb563d44 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:41.886077 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1291/2572] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:55:42.443938 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea646572-6e79-4624-b74f-802d74c5a305 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:42.445806 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ea646572-6e79-4624-b74f-802d74c5a305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:42.459051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea646572-6e79-4624-b74f-802d74c5a305 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:42.465676 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:42.466648 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:42.467215 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:42.467576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8 admin admin] PATH_INFO: `/v3/projects/8abfa26056a84b49a9a52230e81f8a1d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:42.469037 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8abfa26056a84b49a9a52230e81f8a1d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:42.471293 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:42.530354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8abfa26056a84b49a9a52230e81f8a1d', 'request_id': 'req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:42.537560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8abfa26056a84b49a9a52230e81f8a1d', 'request_id': 'req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:42.546266 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8abfa26056a84b49a9a52230e81f8a1d', 'request_id': 'req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:42.557674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8 admin admin] The token cache is being invalidate because project 8abfa26056a84b49a9a52230e81f8a1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:55:42.557978 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:42.558990 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:42.560638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1282/2573] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:55:42 2022] DELETE /identity/v3/projects/8abfa26056a84b49a9a52230e81f8a1d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:42.566053 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cc61e-7809-4ea1-b26f-741acb563d44 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:42.582907 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cc61e-7809-4ea1-b26f-741acb563d44 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:42.605977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:42.651214 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068272f-b7ff-4270-9cea-6ebb1ad3a237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:42.651874 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068272f-b7ff-4270-9cea-6ebb1ad3a237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:42.652156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068272f-b7ff-4270-9cea-6ebb1ad3a237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:42.652429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068272f-b7ff-4270-9cea-6ebb1ad3a237 admin admin] PATH_INFO: `/v3/users/3ff14623626d48689d0003e1de51bbe7` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:42.653672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068272f-b7ff-4270-9cea-6ebb1ad3a237 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ff14623626d48689d0003e1de51bbe7)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:42.655600 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068272f-b7ff-4270-9cea-6ebb1ad3a237 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:42.707539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-4068272f-b7ff-4270-9cea-6ebb1ad3a237 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ff14623626d48689d0003e1de51bbe7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:42.716104 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-4068272f-b7ff-4270-9cea-6ebb1ad3a237 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ff14623626d48689d0003e1de51bbe7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:42.717019 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-4068272f-b7ff-4270-9cea-6ebb1ad3a237 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ff14623626d48689d0003e1de51bbe7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:42.717822 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-4068272f-b7ff-4270-9cea-6ebb1ad3a237 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ff14623626d48689d0003e1de51bbe7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:42.728538 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-4068272f-b7ff-4270-9cea-6ebb1ad3a237 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ff14623626d48689d0003e1de51bbe7'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:42.750827 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1292/2574] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:55:42 2022] DELETE /identity/v3/users/3ff14623626d48689d0003e1de51bbe7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:43.068116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:43.069680 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:43.085278 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09968f4f-1366-49e1-ba8e-57f56f8b2ad8 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:43.166862 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd2230-46da-4c25-8af0-0ce1af2483aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:43.167526 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd2230-46da-4c25-8af0-0ce1af2483aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:43.167811 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd2230-46da-4c25-8af0-0ce1af2483aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:43.168088 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd2230-46da-4c25-8af0-0ce1af2483aa admin admin] PATH_INFO: `/v3/projects/b3fec316f89e4f6da60966e578e9b4d8` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:43.169332 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd2230-46da-4c25-8af0-0ce1af2483aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3fec316f89e4f6da60966e578e9b4d8)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:43.171352 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd2230-46da-4c25-8af0-0ce1af2483aa admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:43.209158 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:43.231922 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5dcd2230-46da-4c25-8af0-0ce1af2483aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3fec316f89e4f6da60966e578e9b4d8', 'request_id': 'req-5dcd2230-46da-4c25-8af0-0ce1af2483aa'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:43.239406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5dcd2230-46da-4c25-8af0-0ce1af2483aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3fec316f89e4f6da60966e578e9b4d8', 'request_id': 'req-5dcd2230-46da-4c25-8af0-0ce1af2483aa'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:43.249169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5dcd2230-46da-4c25-8af0-0ce1af2483aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3fec316f89e4f6da60966e578e9b4d8', 'request_id': 'req-5dcd2230-46da-4c25-8af0-0ce1af2483aa'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:43.262052 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5dcd2230-46da-4c25-8af0-0ce1af2483aa admin admin] The token cache is being invalidate because project b3fec316f89e4f6da60966e578e9b4d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:55:43.262348 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5dcd2230-46da-4c25-8af0-0ce1af2483aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:43.263016 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5dcd2230-46da-4c25-8af0-0ce1af2483aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:55:43.264558 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1283/2575] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:55:43 2022] DELETE /identity/v3/projects/b3fec316f89e4f6da60966e578e9b4d8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:55:43.273418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c3887-c02f-47e3-afe0-ef9612296beb None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:43.273851 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c3887-c02f-47e3-afe0-ef9612296beb None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:43.274345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c3887-c02f-47e3-afe0-ef9612296beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:43.296611 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-2a0c3887-c02f-47e3-afe0-ef9612296beb None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:43.297903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2a0c3887-c02f-47e3-afe0-ef9612296beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:43.364573 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1293/2576] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:55:43.373423 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc87b9-9fce-48a1-80de-84d8dade4aa4 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:43.373846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc87b9-9fce-48a1-80de-84d8dade4aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:43.374218 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc87b9-9fce-48a1-80de-84d8dade4aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:43.396989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-f0fc87b9-9fce-48a1-80de-84d8dade4aa4 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:43.398249 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f0fc87b9-9fce-48a1-80de-84d8dade4aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:43.406982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1284/2577] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:55:43.411701 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c3887-c02f-47e3-afe0-ef9612296beb None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:43.422451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c3887-c02f-47e3-afe0-ef9612296beb None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:43.429340 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b17e1-fc6f-4a2f-84fa-fd51b53e3805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:43.429997 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b17e1-fc6f-4a2f-84fa-fd51b53e3805 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:43.430272 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b17e1-fc6f-4a2f-84fa-fd51b53e3805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:43.430540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b17e1-fc6f-4a2f-84fa-fd51b53e3805 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:43.431186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b17e1-fc6f-4a2f-84fa-fd51b53e3805 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:55:43.431485 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b17e1-fc6f-4a2f-84fa-fd51b53e3805 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:43.433169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b17e1-fc6f-4a2f-84fa-fd51b53e3805 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:43.446320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1294/2578] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:55:43.451045 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc87b9-9fce-48a1-80de-84d8dade4aa4 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:43.461476 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc87b9-9fce-48a1-80de-84d8dade4aa4 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:43.467067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6e794-015a-4fc4-b4dc-7666e136c471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:43.467920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6e794-015a-4fc4-b4dc-7666e136c471 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:43.468470 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6e794-015a-4fc4-b4dc-7666e136c471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:43.468846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6e794-015a-4fc4-b4dc-7666e136c471 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:43.470024 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb6e794-015a-4fc4-b4dc-7666e136c471 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:43.472841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb6e794-015a-4fc4-b4dc-7666e136c471 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:43.513663 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1285/2579] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:55:43 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:55:43.518140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b17e1-fc6f-4a2f-84fa-fd51b53e3805 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:43.528683 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b17e1-fc6f-4a2f-84fa-fd51b53e3805 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:43.567470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:43.603476 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567d151-fb79-46fe-b520-eb9e8565cd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:43.604346 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567d151-fb79-46fe-b520-eb9e8565cd77 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:43.604658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567d151-fb79-46fe-b520-eb9e8565cd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:43.604902 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567d151-fb79-46fe-b520-eb9e8565cd77 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:43.605643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567d151-fb79-46fe-b520-eb9e8565cd77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:43.607829 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567d151-fb79-46fe-b520-eb9e8565cd77 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:43.631914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1295/2580] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:55:43 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:55:43.636800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6e794-015a-4fc4-b4dc-7666e136c471 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:43.648593 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6e794-015a-4fc4-b4dc-7666e136c471 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:43.653853 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af0acd-8530-43e2-8f18-7e2e1977a8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:43.654475 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af0acd-8530-43e2-8f18-7e2e1977a8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:43.654753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af0acd-8530-43e2-8f18-7e2e1977a8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:43.655025 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af0acd-8530-43e2-8f18-7e2e1977a8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:43.655670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af0acd-8530-43e2-8f18-7e2e1977a8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:43.657339 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af0acd-8530-43e2-8f18-7e2e1977a8e6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:43.668463 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1286/2581] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:55:43.673045 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567d151-fb79-46fe-b520-eb9e8565cd77 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:43.684716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567d151-fb79-46fe-b520-eb9e8565cd77 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:43.690054 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3c768-4283-49e2-a53c-a95cc34b8c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:43.690677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3c768-4283-49e2-a53c-a95cc34b8c2a admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:43.690945 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3c768-4283-49e2-a53c-a95cc34b8c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:43.691211 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3c768-4283-49e2-a53c-a95cc34b8c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:43.691849 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3c768-4283-49e2-a53c-a95cc34b8c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:43.693486 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3c768-4283-49e2-a53c-a95cc34b8c2a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:43.705922 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1296/2582] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:55:43.712238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af0acd-8530-43e2-8f18-7e2e1977a8e6 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:43.726164 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af0acd-8530-43e2-8f18-7e2e1977a8e6 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:43.732298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb25d5-3d50-426f-a3e5-2cda5518e5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:43.733070 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb25d5-3d50-426f-a3e5-2cda5518e5f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:43.733379 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb25d5-3d50-426f-a3e5-2cda5518e5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:43.733654 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb25d5-3d50-426f-a3e5-2cda5518e5f9 admin admin] PATH_INFO: `/v3/projects/e55cc9c3a4f947e48bae44d1169ec3a7/users/3e81c987498e4866aa9cef6bc9d102c6/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:43.750789 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb25d5-3d50-426f-a3e5-2cda5518e5f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e55cc9c3a4f947e48bae44d1169ec3a7, user_id=3e81c987498e4866aa9cef6bc9d102c6, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:43.753613 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb25d5-3d50-426f-a3e5-2cda5518e5f9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:43.774340 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1287/2583] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:55:43 2022] PUT /identity/v3/projects/e55cc9c3a4f947e48bae44d1169ec3a7/users/3e81c987498e4866aa9cef6bc9d102c6/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:55:43.785564 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721f271-e798-4750-81e2-5f8648f6a7c6 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:43.785839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721f271-e798-4750-81e2-5f8648f6a7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:43.786102 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721f271-e798-4750-81e2-5f8648f6a7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:43.818640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-2721f271-e798-4750-81e2-5f8648f6a7c6 None None] MFA Rules not processed for user `3e81c987498e4866aa9cef6bc9d102c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:55:43.819929 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2721f271-e798-4750-81e2-5f8648f6a7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:43.893431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1297/2584] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:55:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:55:43.906424 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb25d5-3d50-426f-a3e5-2cda5518e5f9 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:43.908253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-fcbb25d5-3d50-426f-a3e5-2cda5518e5f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:43.974674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb25d5-3d50-426f-a3e5-2cda5518e5f9 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:43.978596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62117a3-f51c-4018-a16e-f08da9f4e637 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c0fabecff2457e931c8c1c36681197', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:43.979336 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62117a3-f51c-4018-a16e-f08da9f4e637 service glance] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:43.979599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62117a3-f51c-4018-a16e-f08da9f4e637 service glance] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:43.979869 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62117a3-f51c-4018-a16e-f08da9f4e637 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:43.989903 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62117a3-f51c-4018-a16e-f08da9f4e637 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:43.991381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62117a3-f51c-4018-a16e-f08da9f4e637 service glance] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:44.002028 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1288/2585] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:55:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:55:44.723992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721f271-e798-4750-81e2-5f8648f6a7c6 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:55:44.725957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2721f271-e798-4750-81e2-5f8648f6a7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:44.822375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721f271-e798-4750-81e2-5f8648f6a7c6 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:55:44.828753 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c159cc-b79d-4036-8a8b-f8f8f671be05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:55:44.829427 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c159cc-b79d-4036-8a8b-f8f8f671be05 service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:55:44.829704 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c159cc-b79d-4036-8a8b-f8f8f671be05 service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:55:44.829970 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c159cc-b79d-4036-8a8b-f8f8f671be05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:55:44.832412 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-04c159cc-b79d-4036-8a8b-f8f8f671be05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:55:44.876592 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:44.945755 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c159cc-b79d-4036-8a8b-f8f8f671be05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:55:44.947644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c159cc-b79d-4036-8a8b-f8f8f671be05 service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:55:44.974539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1298/2586] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:55:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:55:53.996983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:55:54.971646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=25 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:00.577048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62117a3-f51c-4018-a16e-f08da9f4e637 service glance] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:00.578594 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-f62117a3-f51c-4018-a16e-f08da9f4e637 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:00.610006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62117a3-f51c-4018-a16e-f08da9f4e637 service glance] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:00.616980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe08924-b7de-40e7-8f14-7a0878f072f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:00.617864 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe08924-b7de-40e7-8f14-7a0878f072f0 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:00.618965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe08924-b7de-40e7-8f14-7a0878f072f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:00.619446 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe08924-b7de-40e7-8f14-7a0878f072f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:00.622725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cfe08924-b7de-40e7-8f14-7a0878f072f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:00.640119 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe08924-b7de-40e7-8f14-7a0878f072f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:00.642571 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe08924-b7de-40e7-8f14-7a0878f072f0 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:00.672112 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1289/2587] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:06.038843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c159cc-b79d-4036-8a8b-f8f8f671be05 service nova] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:06.040394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-04c159cc-b79d-4036-8a8b-f8f8f671be05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:06.104126 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c159cc-b79d-4036-8a8b-f8f8f671be05 service nova] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:06.109726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30a93d-b431-46a2-bfff-0eb3f24d55aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:06.110521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30a93d-b431-46a2-bfff-0eb3f24d55aa service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:06.110872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30a93d-b431-46a2-bfff-0eb3f24d55aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:06.111191 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30a93d-b431-46a2-bfff-0eb3f24d55aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:06.113916 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9f30a93d-b431-46a2-bfff-0eb3f24d55aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:06.177542 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30a93d-b431-46a2-bfff-0eb3f24d55aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:06.178970 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30a93d-b431-46a2-bfff-0eb3f24d55aa service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:06.189127 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1299/2588] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:06.561096 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe08924-b7de-40e7-8f14-7a0878f072f0 service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:06.562870 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cfe08924-b7de-40e7-8f14-7a0878f072f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:06.575994 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe08924-b7de-40e7-8f14-7a0878f072f0 service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:06.581157 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd612bbd-90b2-4b61-b8df-913dece4ec79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:06.581405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd612bbd-90b2-4b61-b8df-913dece4ec79 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:06.581624 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd612bbd-90b2-4b61-b8df-913dece4ec79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:06.581962 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd612bbd-90b2-4b61-b8df-913dece4ec79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:06.583775 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-fd612bbd-90b2-4b61-b8df-913dece4ec79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:06.596952 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd612bbd-90b2-4b61-b8df-913dece4ec79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:06.598430 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd612bbd-90b2-4b61-b8df-913dece4ec79 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:06.609219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1290/2589] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:10.536267 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30a93d-b431-46a2-bfff-0eb3f24d55aa service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:10.538662 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9f30a93d-b431-46a2-bfff-0eb3f24d55aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:10.556980 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30a93d-b431-46a2-bfff-0eb3f24d55aa service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:10.563237 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3dd880-070f-4d8e-aa68-452d5b1c696f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:10.563900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3dd880-070f-4d8e-aa68-452d5b1c696f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:10.564252 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3dd880-070f-4d8e-aa68-452d5b1c696f admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:10.564595 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3dd880-070f-4d8e-aa68-452d5b1c696f admin admin] PATH_INFO: `/v3/users/9d9e9c8e26774e739695b7e4d42eeafd` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:10.567097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3dd880-070f-4d8e-aa68-452d5b1c696f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d9e9c8e26774e739695b7e4d42eeafd)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:10.570225 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3dd880-070f-4d8e-aa68-452d5b1c696f admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:10.645658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7a3dd880-070f-4d8e-aa68-452d5b1c696f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d9e9c8e26774e739695b7e4d42eeafd'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:10.658353 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7a3dd880-070f-4d8e-aa68-452d5b1c696f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d9e9c8e26774e739695b7e4d42eeafd'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:10.659404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7a3dd880-070f-4d8e-aa68-452d5b1c696f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d9e9c8e26774e739695b7e4d42eeafd'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:10.661364 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7a3dd880-070f-4d8e-aa68-452d5b1c696f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d9e9c8e26774e739695b7e4d42eeafd'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:10.682021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7a3dd880-070f-4d8e-aa68-452d5b1c696f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d9e9c8e26774e739695b7e4d42eeafd'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:10.730558 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1300/2590] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:10 2022] DELETE /identity/v3/users/9d9e9c8e26774e739695b7e4d42eeafd => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:10.995416 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd612bbd-90b2-4b61-b8df-913dece4ec79 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:11.015753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd612bbd-90b2-4b61-b8df-913dece4ec79 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:11.067186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3dd880-070f-4d8e-aa68-452d5b1c696f admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:11.068968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7a3dd880-070f-4d8e-aa68-452d5b1c696f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:11.113460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:11.136029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5f7b3-be56-4ac5-9384-5c134ce20683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:11.136725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5f7b3-be56-4ac5-9384-5c134ce20683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.137004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5f7b3-be56-4ac5-9384-5c134ce20683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.137407 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5f7b3-be56-4ac5-9384-5c134ce20683 admin admin] PATH_INFO: `/v3/projects/25d426f519e8498e80c6c85158bd9042` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.138433 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5f7b3-be56-4ac5-9384-5c134ce20683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25d426f519e8498e80c6c85158bd9042)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:11.140351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5f7b3-be56-4ac5-9384-5c134ce20683 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:11.161183 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3dd880-070f-4d8e-aa68-452d5b1c696f admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:11.166311 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8fddd-8381-4232-9271-20099d5dadf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:11.167011 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8fddd-8381-4232-9271-20099d5dadf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.167350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8fddd-8381-4232-9271-20099d5dadf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.167621 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8fddd-8381-4232-9271-20099d5dadf9 admin admin] PATH_INFO: `/v3/users/bfaf6a2b8a2348b4aa603cda5fe624f5` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.168744 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8fddd-8381-4232-9271-20099d5dadf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfaf6a2b8a2348b4aa603cda5fe624f5)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:11.170885 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8fddd-8381-4232-9271-20099d5dadf9 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:11.188810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e2a5f7b3-be56-4ac5-9384-5c134ce20683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25d426f519e8498e80c6c85158bd9042', 'request_id': 'req-e2a5f7b3-be56-4ac5-9384-5c134ce20683'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:11.195358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e2a5f7b3-be56-4ac5-9384-5c134ce20683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25d426f519e8498e80c6c85158bd9042', 'request_id': 'req-e2a5f7b3-be56-4ac5-9384-5c134ce20683'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:11.204836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e2a5f7b3-be56-4ac5-9384-5c134ce20683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25d426f519e8498e80c6c85158bd9042', 'request_id': 'req-e2a5f7b3-be56-4ac5-9384-5c134ce20683'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:11.220578 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e2a5f7b3-be56-4ac5-9384-5c134ce20683 admin admin] The token cache is being invalidate because project 25d426f519e8498e80c6c85158bd9042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:11.220872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e2a5f7b3-be56-4ac5-9384-5c134ce20683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:11.221534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-e2a5f7b3-be56-4ac5-9384-5c134ce20683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:11.223069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1291/2591] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:10 2022] DELETE /identity/v3/projects/25d426f519e8498e80c6c85158bd9042 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:11.228006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcebdf2-ee79-4efd-a98b-7a57e0fc6350 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.228192 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-42a8fddd-8381-4232-9271-20099d5dadf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfaf6a2b8a2348b4aa603cda5fe624f5'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:11.228296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcebdf2-ee79-4efd-a98b-7a57e0fc6350 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.228597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcebdf2-ee79-4efd-a98b-7a57e0fc6350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.238136 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-42a8fddd-8381-4232-9271-20099d5dadf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfaf6a2b8a2348b4aa603cda5fe624f5'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:11.238744 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-42a8fddd-8381-4232-9271-20099d5dadf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfaf6a2b8a2348b4aa603cda5fe624f5'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:11.239544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-42a8fddd-8381-4232-9271-20099d5dadf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfaf6a2b8a2348b4aa603cda5fe624f5'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:11.247350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-42a8fddd-8381-4232-9271-20099d5dadf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfaf6a2b8a2348b4aa603cda5fe624f5'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:11.251161 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-cdcebdf2-ee79-4efd-a98b-7a57e0fc6350 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:11.252494 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-cdcebdf2-ee79-4efd-a98b-7a57e0fc6350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:11.265392 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1301/2592] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:11 2022] DELETE /identity/v3/users/bfaf6a2b8a2348b4aa603cda5fe624f5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:11.311430 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1292/2593] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:11.319671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e89ae-3c9b-4483-ac22-39836286fe38 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.320154 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e89ae-3c9b-4483-ac22-39836286fe38 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.320283 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e89ae-3c9b-4483-ac22-39836286fe38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.340843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d88e89ae-3c9b-4483-ac22-39836286fe38 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:11.342219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d88e89ae-3c9b-4483-ac22-39836286fe38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:11.408819 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1302/2594] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:11.413349 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcebdf2-ee79-4efd-a98b-7a57e0fc6350 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:11.419535 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:11.423753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcebdf2-ee79-4efd-a98b-7a57e0fc6350 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:11.433901 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008eefd5-bd3c-475c-8e0f-3f03d28cbd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:11.434821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eefd5-bd3c-475c-8e0f-3f03d28cbd2a admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.435100 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eefd5-bd3c-475c-8e0f-3f03d28cbd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.435389 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eefd5-bd3c-475c-8e0f-3f03d28cbd2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.436218 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008eefd5-bd3c-475c-8e0f-3f03d28cbd2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:56:11.436524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008eefd5-bd3c-475c-8e0f-3f03d28cbd2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:11.438239 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008eefd5-bd3c-475c-8e0f-3f03d28cbd2a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:11.452437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1293/2595] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:56:11.453252 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e89ae-3c9b-4483-ac22-39836286fe38 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:11.454843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d88e89ae-3c9b-4483-ac22-39836286fe38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:11.456897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008eefd5-bd3c-475c-8e0f-3f03d28cbd2a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:11.467715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e89ae-3c9b-4483-ac22-39836286fe38 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:11.469085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008eefd5-bd3c-475c-8e0f-3f03d28cbd2a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:11.473266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d13ec4-52dd-4fde-951b-b37a39540005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:11.473904 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d13ec4-52dd-4fde-951b-b37a39540005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.474177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d13ec4-52dd-4fde-951b-b37a39540005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.474441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d13ec4-52dd-4fde-951b-b37a39540005 admin admin] PATH_INFO: `/v3/projects/3fa25d71f2194e638989c1e1d9ef30da` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.475527 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68023e1e-ef55-4879-96cc-dd107675060d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:11.475672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d13ec4-52dd-4fde-951b-b37a39540005 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fa25d71f2194e638989c1e1d9ef30da)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:11.476296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68023e1e-ef55-4879-96cc-dd107675060d admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.476641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68023e1e-ef55-4879-96cc-dd107675060d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.476972 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68023e1e-ef55-4879-96cc-dd107675060d admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.477457 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d13ec4-52dd-4fde-951b-b37a39540005 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:11.477773 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68023e1e-ef55-4879-96cc-dd107675060d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:11.480563 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68023e1e-ef55-4879-96cc-dd107675060d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:11.514779 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1294/2596] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:11 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:11.522260 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68023e1e-ef55-4879-96cc-dd107675060d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:11.537925 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:11.549207 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-25d13ec4-52dd-4fde-951b-b37a39540005 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fa25d71f2194e638989c1e1d9ef30da', 'request_id': 'req-25d13ec4-52dd-4fde-951b-b37a39540005'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:11.556007 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-25d13ec4-52dd-4fde-951b-b37a39540005 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fa25d71f2194e638989c1e1d9ef30da', 'request_id': 'req-25d13ec4-52dd-4fde-951b-b37a39540005'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:11.564689 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-25d13ec4-52dd-4fde-951b-b37a39540005 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fa25d71f2194e638989c1e1d9ef30da', 'request_id': 'req-25d13ec4-52dd-4fde-951b-b37a39540005'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:11.576082 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-25d13ec4-52dd-4fde-951b-b37a39540005 admin admin] The token cache is being invalidate because project 3fa25d71f2194e638989c1e1d9ef30da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:11.576373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-25d13ec4-52dd-4fde-951b-b37a39540005 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:11.577030 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-25d13ec4-52dd-4fde-951b-b37a39540005 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:11.578599 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1303/2597] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:11 2022] DELETE /identity/v3/projects/3fa25d71f2194e638989c1e1d9ef30da => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:11.584319 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58087b7-543b-4cdc-8f8c-298a301bb898 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.584631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58087b7-543b-4cdc-8f8c-298a301bb898 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.584903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58087b7-543b-4cdc-8f8c-298a301bb898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.593515 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68023e1e-ef55-4879-96cc-dd107675060d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:11.606836 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-e58087b7-543b-4cdc-8f8c-298a301bb898 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:11.608142 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-e58087b7-543b-4cdc-8f8c-298a301bb898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:11.651719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff63a3-bbf9-45b4-8c1d-cb74191be133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:11.652645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff63a3-bbf9-45b4-8c1d-cb74191be133 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.653098 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff63a3-bbf9-45b4-8c1d-cb74191be133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.653406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff63a3-bbf9-45b4-8c1d-cb74191be133 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.654477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff63a3-bbf9-45b4-8c1d-cb74191be133 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:11.656125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff63a3-bbf9-45b4-8c1d-cb74191be133 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:11.672144 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1304/2598] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:11.679138 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1295/2599] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:11 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:11.680949 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b72f7-435a-4291-b34e-d41a214ca03b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.681318 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b72f7-435a-4291-b34e-d41a214ca03b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.681690 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b72f7-435a-4291-b34e-d41a214ca03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.682915 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff63a3-bbf9-45b4-8c1d-cb74191be133 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:11.684668 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-02ff63a3-bbf9-45b4-8c1d-cb74191be133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:11.697109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff63a3-bbf9-45b4-8c1d-cb74191be133 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:11.702066 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bc7e0-6568-4281-92ab-22a23a2b92aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:11.702679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bc7e0-6568-4281-92ab-22a23a2b92aa admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.702953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bc7e0-6568-4281-92ab-22a23a2b92aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.703070 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-4d2b72f7-435a-4291-b34e-d41a214ca03b None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:11.703226 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bc7e0-6568-4281-92ab-22a23a2b92aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.703923 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bc7e0-6568-4281-92ab-22a23a2b92aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:11.704174 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4d2b72f7-435a-4291-b34e-d41a214ca03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:11.705960 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bc7e0-6568-4281-92ab-22a23a2b92aa admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:11.713302 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1305/2600] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:11.716805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1296/2601] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:11.717432 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b72f7-435a-4291-b34e-d41a214ca03b None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:11.720638 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bc7e0-6568-4281-92ab-22a23a2b92aa admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:11.726639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b72f7-435a-4291-b34e-d41a214ca03b None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:11.730700 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bc7e0-6568-4281-92ab-22a23a2b92aa admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:11.731467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a2f73-3296-41a6-8d62-a8e218279a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:11.732067 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a2f73-3296-41a6-8d62-a8e218279a24 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.732343 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a2f73-3296-41a6-8d62-a8e218279a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.732615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a2f73-3296-41a6-8d62-a8e218279a24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.733292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a2f73-3296-41a6-8d62-a8e218279a24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:56:11.733591 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a2f73-3296-41a6-8d62-a8e218279a24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:11.735178 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a2f73-3296-41a6-8d62-a8e218279a24 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:11.736001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2f3dd-0b3e-4eed-aaea-702601a98909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:11.736833 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2f3dd-0b3e-4eed-aaea-702601a98909 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.737119 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2f3dd-0b3e-4eed-aaea-702601a98909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.738032 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2f3dd-0b3e-4eed-aaea-702601a98909 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.738694 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2f3dd-0b3e-4eed-aaea-702601a98909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:11.740403 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2f3dd-0b3e-4eed-aaea-702601a98909 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:11.747484 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1306/2602] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:56:11.752172 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a2f73-3296-41a6-8d62-a8e218279a24 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:11.753557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1297/2603] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:11.757933 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2f3dd-0b3e-4eed-aaea-702601a98909 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:11.763532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a2f73-3296-41a6-8d62-a8e218279a24 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:11.768124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2f3dd-0b3e-4eed-aaea-702601a98909 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:11.768533 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da4323-4fea-4284-b4a3-eb8cb645b828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:11.769266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da4323-4fea-4284-b4a3-eb8cb645b828 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.769564 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da4323-4fea-4284-b4a3-eb8cb645b828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.769914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da4323-4fea-4284-b4a3-eb8cb645b828 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.770485 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da4323-4fea-4284-b4a3-eb8cb645b828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:11.772372 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da4323-4fea-4284-b4a3-eb8cb645b828 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:11.773091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45f264-b77c-4f1b-82e6-ba09676abc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:11.773765 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45f264-b77c-4f1b-82e6-ba09676abc9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.774042 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45f264-b77c-4f1b-82e6-ba09676abc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.774308 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45f264-b77c-4f1b-82e6-ba09676abc9d admin admin] PATH_INFO: `/v3/projects/af67c46729744f6fa0a469ddcf55f47b/users/34f8e59ebe064983a4bb3411d715e66c/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.790006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45f264-b77c-4f1b-82e6-ba09676abc9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=af67c46729744f6fa0a469ddcf55f47b, user_id=34f8e59ebe064983a4bb3411d715e66c, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:11.793201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45f264-b77c-4f1b-82e6-ba09676abc9d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:11.802027 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1307/2604] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:11 2022] POST /identity/v3/projects => generated 355 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:11.806722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da4323-4fea-4284-b4a3-eb8cb645b828 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:11.807389 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1298/2605] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:11 2022] PUT /identity/v3/projects/af67c46729744f6fa0a469ddcf55f47b/users/34f8e59ebe064983a4bb3411d715e66c/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:11.818745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2984a-02d6-48c0-b467-cf8f88cd9319 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.819030 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2984a-02d6-48c0-b467-cf8f88cd9319 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.819314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2984a-02d6-48c0-b467-cf8f88cd9319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.846984 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-5ab2984a-02d6-48c0-b467-cf8f88cd9319 None None] MFA Rules not processed for user `34f8e59ebe064983a4bb3411d715e66c`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:11.848188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5ab2984a-02d6-48c0-b467-cf8f88cd9319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:11.856369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:11.873491 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da4323-4fea-4284-b4a3-eb8cb645b828 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:11.903455 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:11.917064 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1299/2606] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:11.928346 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2984a-02d6-48c0-b467-cf8f88cd9319 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:11.929940 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5ab2984a-02d6-48c0-b467-cf8f88cd9319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:11.936170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84c416-2654-47a9-b65e-569abcfb01bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:11.936757 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84c416-2654-47a9-b65e-569abcfb01bd admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.937033 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84c416-2654-47a9-b65e-569abcfb01bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.937342 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84c416-2654-47a9-b65e-569abcfb01bd admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.937986 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd84c416-2654-47a9-b65e-569abcfb01bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:11.939885 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd84c416-2654-47a9-b65e-569abcfb01bd admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:11.964056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1308/2607] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:11 2022] POST /identity/v3/users => generated 278 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:11.968451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84c416-2654-47a9-b65e-569abcfb01bd admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:11.979398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84c416-2654-47a9-b65e-569abcfb01bd admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:11.985024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02f83e-b03e-47a3-8899-2dc030cff90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:11.985659 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02f83e-b03e-47a3-8899-2dc030cff90a admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:11.985942 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02f83e-b03e-47a3-8899-2dc030cff90a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:11.986236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02f83e-b03e-47a3-8899-2dc030cff90a admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:11.986880 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f02f83e-b03e-47a3-8899-2dc030cff90a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:11.988535 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f02f83e-b03e-47a3-8899-2dc030cff90a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:12.002221 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1309/2608] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:12.003648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02f83e-b03e-47a3-8899-2dc030cff90a admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:12.006588 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2984a-02d6-48c0-b467-cf8f88cd9319 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:12.013086 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7534791-c4ae-4b51-815b-1e3707ed6515 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:12.013727 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7534791-c4ae-4b51-815b-1e3707ed6515 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:12.014004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7534791-c4ae-4b51-815b-1e3707ed6515 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:12.014122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02f83e-b03e-47a3-8899-2dc030cff90a admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:12.014272 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7534791-c4ae-4b51-815b-1e3707ed6515 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:12.019920 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7fdaf-e9af-4a1e-ab94-3674625ec2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:12.021008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7fdaf-e9af-4a1e-ab94-3674625ec2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:12.021316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7fdaf-e9af-4a1e-ab94-3674625ec2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:12.021588 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7fdaf-e9af-4a1e-ab94-3674625ec2c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:12.022399 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7fdaf-e9af-4a1e-ab94-3674625ec2c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:12.022687 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7534791-c4ae-4b51-815b-1e3707ed6515 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:12.024023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7fdaf-e9af-4a1e-ab94-3674625ec2c4 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:12.024291 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7534791-c4ae-4b51-815b-1e3707ed6515 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:12.034743 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1300/2609] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:56:12.036298 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1310/2610] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:12.041066 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7534791-c4ae-4b51-815b-1e3707ed6515 service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:12.051259 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7534791-c4ae-4b51-815b-1e3707ed6515 service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:12.056566 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa397d7c-f084-4793-8890-228afc9ab0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:12.057363 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa397d7c-f084-4793-8890-228afc9ab0f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:12.058046 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa397d7c-f084-4793-8890-228afc9ab0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:12.058359 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa397d7c-f084-4793-8890-228afc9ab0f3 admin admin] PATH_INFO: `/v3/projects/eb022978bd494438bf818c7543cd1733/users/aaac001a38a74cee9eaeb49d6b5900f8/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:12.074724 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa397d7c-f084-4793-8890-228afc9ab0f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb022978bd494438bf818c7543cd1733, user_id=aaac001a38a74cee9eaeb49d6b5900f8, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:12.077401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa397d7c-f084-4793-8890-228afc9ab0f3 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:12.093085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1301/2611] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:12 2022] PUT /identity/v3/projects/eb022978bd494438bf818c7543cd1733/users/aaac001a38a74cee9eaeb49d6b5900f8/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:12.103795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090507a-6c9f-479d-8981-0a237e24df9c None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:12.104091 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090507a-6c9f-479d-8981-0a237e24df9c None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:12.104366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090507a-6c9f-479d-8981-0a237e24df9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:12.133342 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-c090507a-6c9f-479d-8981-0a237e24df9c None None] MFA Rules not processed for user `aaac001a38a74cee9eaeb49d6b5900f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:12.134746 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c090507a-6c9f-479d-8981-0a237e24df9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:12.204427 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1311/2612] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:12.209886 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa397d7c-f084-4793-8890-228afc9ab0f3 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:12.280454 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa397d7c-f084-4793-8890-228afc9ab0f3 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:12.285622 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece04c09-5895-4ae1-a120-7ed9f9be5640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:12.286236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece04c09-5895-4ae1-a120-7ed9f9be5640 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:12.286511 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece04c09-5895-4ae1-a120-7ed9f9be5640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:12.286780 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece04c09-5895-4ae1-a120-7ed9f9be5640 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:12.287414 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece04c09-5895-4ae1-a120-7ed9f9be5640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:12.289382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece04c09-5895-4ae1-a120-7ed9f9be5640 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:12.310999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:12.333328 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1302/2613] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:12 2022] POST /identity/v3/projects => generated 353 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:12.334810 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090507a-6c9f-479d-8981-0a237e24df9c None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:12.350176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090507a-6c9f-479d-8981-0a237e24df9c None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:12.444881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcf4e8-6057-456c-bf88-3148740d7a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:12.445612 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcf4e8-6057-456c-bf88-3148740d7a70 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:12.445932 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcf4e8-6057-456c-bf88-3148740d7a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:12.446265 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcf4e8-6057-456c-bf88-3148740d7a70 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:12.446953 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcf4e8-6057-456c-bf88-3148740d7a70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:12.449071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcf4e8-6057-456c-bf88-3148740d7a70 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:12.472844 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1312/2614] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:12 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:12.477202 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece04c09-5895-4ae1-a120-7ed9f9be5640 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:12.487829 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece04c09-5895-4ae1-a120-7ed9f9be5640 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:12.493485 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd8d12-f43f-486f-8753-ab4b078abee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:12.494294 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd8d12-f43f-486f-8753-ab4b078abee7 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:12.494879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd8d12-f43f-486f-8753-ab4b078abee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:12.495253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd8d12-f43f-486f-8753-ab4b078abee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:12.496346 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd8d12-f43f-486f-8753-ab4b078abee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:12.498972 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd8d12-f43f-486f-8753-ab4b078abee7 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:12.512570 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1303/2615] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:12.518658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcf4e8-6057-456c-bf88-3148740d7a70 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:12.529807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcf4e8-6057-456c-bf88-3148740d7a70 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:12.535718 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620eccc-52c1-4aa9-8b6b-15cba14c2801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:12.536342 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620eccc-52c1-4aa9-8b6b-15cba14c2801 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:12.536624 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620eccc-52c1-4aa9-8b6b-15cba14c2801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:12.536896 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620eccc-52c1-4aa9-8b6b-15cba14c2801 admin admin] PATH_INFO: `/v3/projects/78b3cab8e7094e8d8c07643a948c6b46/users/da297351b65f4fa1b31bf660b1399bbd/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:12.552407 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620eccc-52c1-4aa9-8b6b-15cba14c2801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78b3cab8e7094e8d8c07643a948c6b46, user_id=da297351b65f4fa1b31bf660b1399bbd, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:12.555162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620eccc-52c1-4aa9-8b6b-15cba14c2801 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:12.557487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:12.577080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1313/2616] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:12 2022] PUT /identity/v3/projects/78b3cab8e7094e8d8c07643a948c6b46/users/da297351b65f4fa1b31bf660b1399bbd/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:12.582419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd8d12-f43f-486f-8753-ab4b078abee7 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:12.593034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd8d12-f43f-486f-8753-ab4b078abee7 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:12.651582 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22ff5d-a987-40f3-987d-4d551302d6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:12.652193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22ff5d-a987-40f3-987d-4d551302d6a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:12.652465 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22ff5d-a987-40f3-987d-4d551302d6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:12.652740 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22ff5d-a987-40f3-987d-4d551302d6a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:12.653412 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22ff5d-a987-40f3-987d-4d551302d6a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:12.655010 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22ff5d-a987-40f3-987d-4d551302d6a4 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:12.665014 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1304/2617] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:12.669460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620eccc-52c1-4aa9-8b6b-15cba14c2801 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:12.680532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620eccc-52c1-4aa9-8b6b-15cba14c2801 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:12.685313 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a80f2-5249-45a0-ad16-754ab2427757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:12.685957 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a80f2-5249-45a0-ad16-754ab2427757 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:12.686228 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a80f2-5249-45a0-ad16-754ab2427757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:12.686473 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a80f2-5249-45a0-ad16-754ab2427757 admin admin] PATH_INFO: `/v3/projects/78b3cab8e7094e8d8c07643a948c6b46/users/da297351b65f4fa1b31bf660b1399bbd/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:12.688406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a80f2-5249-45a0-ad16-754ab2427757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78b3cab8e7094e8d8c07643a948c6b46, user_id=da297351b65f4fa1b31bf660b1399bbd, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:12.691060 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a80f2-5249-45a0-ad16-754ab2427757 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:12.701528 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1314/2618] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:12 2022] PUT /identity/v3/projects/78b3cab8e7094e8d8c07643a948c6b46/users/da297351b65f4fa1b31bf660b1399bbd/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:12.714208 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e76a6-c673-44b7-bc89-0ecbd4d367c1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:12.714534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e76a6-c673-44b7-bc89-0ecbd4d367c1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:12.714903 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e76a6-c673-44b7-bc89-0ecbd4d367c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:12.749284 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-de0e76a6-c673-44b7-bc89-0ecbd4d367c1 None None] MFA Rules not processed for user `da297351b65f4fa1b31bf660b1399bbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:12.750527 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-de0e76a6-c673-44b7-bc89-0ecbd4d367c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:12.824569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1305/2619] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:12 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:12.836913 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a80f2-5249-45a0-ad16-754ab2427757 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:12.839040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-009a80f2-5249-45a0-ad16-754ab2427757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:12.910962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a80f2-5249-45a0-ad16-754ab2427757 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:12.916036 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed1182-97ca-444d-95ec-03b8c0e84638 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:12.916658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed1182-97ca-444d-95ec-03b8c0e84638 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:12.916934 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed1182-97ca-444d-95ec-03b8c0e84638 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:12.917226 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed1182-97ca-444d-95ec-03b8c0e84638 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:12.927709 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed1182-97ca-444d-95ec-03b8c0e84638 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:12.929746 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed1182-97ca-444d-95ec-03b8c0e84638 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:12.939460 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1315/2620] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:12 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:13.020452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e76a6-c673-44b7-bc89-0ecbd4d367c1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:13.030790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e76a6-c673-44b7-bc89-0ecbd4d367c1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:13.034853 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32951f06-b702-4185-9bf2-b0472614514a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:13.035450 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32951f06-b702-4185-9bf2-b0472614514a service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:13.035726 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32951f06-b702-4185-9bf2-b0472614514a service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:13.035992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32951f06-b702-4185-9bf2-b0472614514a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:13.097762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32951f06-b702-4185-9bf2-b0472614514a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:13.099369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32951f06-b702-4185-9bf2-b0472614514a service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:13.109704 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1306/2621] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:13 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:22.935009 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:INSERT=2,keystone:SELECT=33 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:23.104828 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=98 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:24.739609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed1182-97ca-444d-95ec-03b8c0e84638 service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:24.821598 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed1182-97ca-444d-95ec-03b8c0e84638 service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:24.827123 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020589e-33f4-4a18-9c0e-b14c55a95fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:24.827755 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020589e-33f4-4a18-9c0e-b14c55a95fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:24.828031 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020589e-33f4-4a18-9c0e-b14c55a95fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:24.828310 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020589e-33f4-4a18-9c0e-b14c55a95fe7 admin admin] PATH_INFO: `/v3/users/34f8e59ebe064983a4bb3411d715e66c` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:24.829879 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020589e-33f4-4a18-9c0e-b14c55a95fe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34f8e59ebe064983a4bb3411d715e66c)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:24.831796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020589e-33f4-4a18-9c0e-b14c55a95fe7 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:24.882795 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5020589e-33f4-4a18-9c0e-b14c55a95fe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34f8e59ebe064983a4bb3411d715e66c'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:24.890693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5020589e-33f4-4a18-9c0e-b14c55a95fe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34f8e59ebe064983a4bb3411d715e66c'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:24.891304 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5020589e-33f4-4a18-9c0e-b14c55a95fe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34f8e59ebe064983a4bb3411d715e66c'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:24.892048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5020589e-33f4-4a18-9c0e-b14c55a95fe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34f8e59ebe064983a4bb3411d715e66c'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:24.899084 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-5020589e-33f4-4a18-9c0e-b14c55a95fe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34f8e59ebe064983a4bb3411d715e66c'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:24.915852 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1316/2622] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:24 2022] DELETE /identity/v3/users/34f8e59ebe064983a4bb3411d715e66c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:24.920519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e728a-9ec9-4c59-a47f-7ff6f500247a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:24.921031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e728a-9ec9-4c59-a47f-7ff6f500247a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:24.921031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e728a-9ec9-4c59-a47f-7ff6f500247a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:24.942217 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-4a2e728a-9ec9-4c59-a47f-7ff6f500247a None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:24.943430 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4a2e728a-9ec9-4c59-a47f-7ff6f500247a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:25.011806 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1307/2623] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:25.022976 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020589e-33f4-4a18-9c0e-b14c55a95fe7 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:25.024864 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5020589e-33f4-4a18-9c0e-b14c55a95fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:25.035514 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e728a-9ec9-4c59-a47f-7ff6f500247a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:25.037077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4a2e728a-9ec9-4c59-a47f-7ff6f500247a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:25.049584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e728a-9ec9-4c59-a47f-7ff6f500247a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:25.054331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ce5c5-b963-4ba9-ac0a-99c48bb446f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:25.054958 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ce5c5-b963-4ba9-ac0a-99c48bb446f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:25.055219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ce5c5-b963-4ba9-ac0a-99c48bb446f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:25.055489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ce5c5-b963-4ba9-ac0a-99c48bb446f6 admin admin] PATH_INFO: `/v3/users/3e81c987498e4866aa9cef6bc9d102c6` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:25.056631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ce5c5-b963-4ba9-ac0a-99c48bb446f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e81c987498e4866aa9cef6bc9d102c6)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:25.058659 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ce5c5-b963-4ba9-ac0a-99c48bb446f6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:25.092458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020589e-33f4-4a18-9c0e-b14c55a95fe7 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:25.096698 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08231ba4-0c5e-4c88-a154-324cae3e369f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:25.097540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08231ba4-0c5e-4c88-a154-324cae3e369f service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:25.097806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08231ba4-0c5e-4c88-a154-324cae3e369f service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:25.098220 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08231ba4-0c5e-4c88-a154-324cae3e369f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:25.107722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08231ba4-0c5e-4c88-a154-324cae3e369f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:25.108967 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-755ce5c5-b963-4ba9-ac0a-99c48bb446f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e81c987498e4866aa9cef6bc9d102c6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:25.109178 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08231ba4-0c5e-4c88-a154-324cae3e369f service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:25.117685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-755ce5c5-b963-4ba9-ac0a-99c48bb446f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e81c987498e4866aa9cef6bc9d102c6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:25.118386 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-755ce5c5-b963-4ba9-ac0a-99c48bb446f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e81c987498e4866aa9cef6bc9d102c6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:25.118999 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-755ce5c5-b963-4ba9-ac0a-99c48bb446f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e81c987498e4866aa9cef6bc9d102c6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:25.120383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1317/2624] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:25.132038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-755ce5c5-b963-4ba9-ac0a-99c48bb446f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e81c987498e4866aa9cef6bc9d102c6'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:25.148562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1308/2625] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:25 2022] DELETE /identity/v3/users/3e81c987498e4866aa9cef6bc9d102c6 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:25.152909 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0326f-0db1-4888-958d-c5be5011af7a None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:25.153248 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0326f-0db1-4888-958d-c5be5011af7a None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:25.153522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0326f-0db1-4888-958d-c5be5011af7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:25.173877 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-16e0326f-0db1-4888-958d-c5be5011af7a None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:25.175197 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-16e0326f-0db1-4888-958d-c5be5011af7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:25.193565 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:25.241976 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1318/2626] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:25.251936 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ce5c5-b963-4ba9-ac0a-99c48bb446f6 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:25.253534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-755ce5c5-b963-4ba9-ac0a-99c48bb446f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:25.319466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ce5c5-b963-4ba9-ac0a-99c48bb446f6 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:25.323569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97880ab8-eb09-48f6-bf35-7c82e73714d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:25.324266 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97880ab8-eb09-48f6-bf35-7c82e73714d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:25.324545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97880ab8-eb09-48f6-bf35-7c82e73714d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:25.324862 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97880ab8-eb09-48f6-bf35-7c82e73714d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:25.341488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97880ab8-eb09-48f6-bf35-7c82e73714d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:25.343812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97880ab8-eb09-48f6-bf35-7c82e73714d6 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:25.356550 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1309/2627] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:25.414690 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0326f-0db1-4888-958d-c5be5011af7a None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:25.416200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-16e0326f-0db1-4888-958d-c5be5011af7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:25.432259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0326f-0db1-4888-958d-c5be5011af7a None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:25.443019 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce7748-45d6-4d37-95f3-f13cf5232d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:25.443681 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce7748-45d6-4d37-95f3-f13cf5232d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:25.443960 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce7748-45d6-4d37-95f3-f13cf5232d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:25.444244 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce7748-45d6-4d37-95f3-f13cf5232d93 admin admin] PATH_INFO: `/v3/projects/af67c46729744f6fa0a469ddcf55f47b` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:25.445439 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce7748-45d6-4d37-95f3-f13cf5232d93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af67c46729744f6fa0a469ddcf55f47b)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:25.447406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce7748-45d6-4d37-95f3-f13cf5232d93 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:25.501027 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-55ce7748-45d6-4d37-95f3-f13cf5232d93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af67c46729744f6fa0a469ddcf55f47b', 'request_id': 'req-55ce7748-45d6-4d37-95f3-f13cf5232d93'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:25.507519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-55ce7748-45d6-4d37-95f3-f13cf5232d93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af67c46729744f6fa0a469ddcf55f47b', 'request_id': 'req-55ce7748-45d6-4d37-95f3-f13cf5232d93'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:25.516545 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-55ce7748-45d6-4d37-95f3-f13cf5232d93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af67c46729744f6fa0a469ddcf55f47b', 'request_id': 'req-55ce7748-45d6-4d37-95f3-f13cf5232d93'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:25.528644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-55ce7748-45d6-4d37-95f3-f13cf5232d93 admin admin] The token cache is being invalidate because project af67c46729744f6fa0a469ddcf55f47b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:25.528932 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-55ce7748-45d6-4d37-95f3-f13cf5232d93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:25.529630 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-55ce7748-45d6-4d37-95f3-f13cf5232d93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:25.531606 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1319/2628] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:25 2022] DELETE /identity/v3/projects/af67c46729744f6fa0a469ddcf55f47b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:25.539049 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cdd19-c76a-45ed-8310-0c2e2ab61449 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:25.539646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cdd19-c76a-45ed-8310-0c2e2ab61449 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:25.540151 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cdd19-c76a-45ed-8310-0c2e2ab61449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:25.576252 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-927cdd19-c76a-45ed-8310-0c2e2ab61449 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:25.577778 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-927cdd19-c76a-45ed-8310-0c2e2ab61449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:25.590370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:25.652319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1310/2629] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:25.662056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f3f0b-85fd-4afa-a409-9d54eaed581e None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:25.662743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f3f0b-85fd-4afa-a409-9d54eaed581e None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:25.663482 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f3f0b-85fd-4afa-a409-9d54eaed581e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:25.687733 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-564f3f0b-85fd-4afa-a409-9d54eaed581e None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:25.689034 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-564f3f0b-85fd-4afa-a409-9d54eaed581e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:25.690924 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cdd19-c76a-45ed-8310-0c2e2ab61449 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:25.692264 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-927cdd19-c76a-45ed-8310-0c2e2ab61449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:25.702468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1320/2630] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:25.707822 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f3f0b-85fd-4afa-a409-9d54eaed581e None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:25.709120 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cdd19-c76a-45ed-8310-0c2e2ab61449 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:25.715103 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629cd40e-cbb2-4975-b2c8-f7907e717bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:25.715847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cd40e-cbb2-4975-b2c8-f7907e717bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:25.716329 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cd40e-cbb2-4975-b2c8-f7907e717bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:25.716602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cd40e-cbb2-4975-b2c8-f7907e717bd0 admin admin] PATH_INFO: `/v3/projects/e55cc9c3a4f947e48bae44d1169ec3a7` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:25.718528 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629cd40e-cbb2-4975-b2c8-f7907e717bd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e55cc9c3a4f947e48bae44d1169ec3a7)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:25.719254 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f3f0b-85fd-4afa-a409-9d54eaed581e None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:25.720908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629cd40e-cbb2-4975-b2c8-f7907e717bd0 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:25.726318 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeceea7-1d46-4a5b-8c8f-16c359c2af9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:25.726827 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeceea7-1d46-4a5b-8c8f-16c359c2af9e admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:25.727270 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeceea7-1d46-4a5b-8c8f-16c359c2af9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:25.727625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeceea7-1d46-4a5b-8c8f-16c359c2af9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:25.728499 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeceea7-1d46-4a5b-8c8f-16c359c2af9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:56:25.729365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeceea7-1d46-4a5b-8c8f-16c359c2af9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:25.732547 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeceea7-1d46-4a5b-8c8f-16c359c2af9e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:25.746501 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1321/2631] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:56:25.750856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeceea7-1d46-4a5b-8c8f-16c359c2af9e admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:25.762109 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeceea7-1d46-4a5b-8c8f-16c359c2af9e admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:25.768782 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6949124-b54d-4be0-a4e3-200798de1114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:25.769582 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6949124-b54d-4be0-a4e3-200798de1114 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:25.769893 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6949124-b54d-4be0-a4e3-200798de1114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:25.770180 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6949124-b54d-4be0-a4e3-200798de1114 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:25.770851 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6949124-b54d-4be0-a4e3-200798de1114 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:25.772761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6949124-b54d-4be0-a4e3-200798de1114 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:25.781893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-629cd40e-cbb2-4975-b2c8-f7907e717bd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e55cc9c3a4f947e48bae44d1169ec3a7', 'request_id': 'req-629cd40e-cbb2-4975-b2c8-f7907e717bd0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:25.790071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-629cd40e-cbb2-4975-b2c8-f7907e717bd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e55cc9c3a4f947e48bae44d1169ec3a7', 'request_id': 'req-629cd40e-cbb2-4975-b2c8-f7907e717bd0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:25.800811 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-629cd40e-cbb2-4975-b2c8-f7907e717bd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e55cc9c3a4f947e48bae44d1169ec3a7', 'request_id': 'req-629cd40e-cbb2-4975-b2c8-f7907e717bd0'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:25.809308 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1322/2632] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:25 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:25.813677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6949124-b54d-4be0-a4e3-200798de1114 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:25.815876 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-629cd40e-cbb2-4975-b2c8-f7907e717bd0 admin admin] The token cache is being invalidate because project e55cc9c3a4f947e48bae44d1169ec3a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:25.816167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-629cd40e-cbb2-4975-b2c8-f7907e717bd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:25.818279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-629cd40e-cbb2-4975-b2c8-f7907e717bd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:25.819983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1311/2633] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:25 2022] DELETE /identity/v3/projects/e55cc9c3a4f947e48bae44d1169ec3a7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:25.824733 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb500b9-b8a7-4298-9734-4001ce9a6fe7 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:25.825048 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb500b9-b8a7-4298-9734-4001ce9a6fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:25.825318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb500b9-b8a7-4298-9734-4001ce9a6fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:25.850101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-deb500b9-b8a7-4298-9734-4001ce9a6fe7 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:25.851409 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-deb500b9-b8a7-4298-9734-4001ce9a6fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:25.904564 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6949124-b54d-4be0-a4e3-200798de1114 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:25.912834 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c3844-16df-4d04-a10b-2514ffd4bf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:25.913416 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c3844-16df-4d04-a10b-2514ffd4bf1c admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:25.913694 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c3844-16df-4d04-a10b-2514ffd4bf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:25.913962 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c3844-16df-4d04-a10b-2514ffd4bf1c admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:25.914605 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c3844-16df-4d04-a10b-2514ffd4bf1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:25.916713 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c3844-16df-4d04-a10b-2514ffd4bf1c admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:25.928796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:25.931374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1312/2634] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:25.939497 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57f21a-221c-4a77-8c4d-d10909432547 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:25.939788 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57f21a-221c-4a77-8c4d-d10909432547 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:25.940076 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57f21a-221c-4a77-8c4d-d10909432547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:25.949256 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1323/2635] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:25 2022] POST /identity/v3/users => generated 283 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:25.953055 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c3844-16df-4d04-a10b-2514ffd4bf1c admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:25.955597 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-fc7c3844-16df-4d04-a10b-2514ffd4bf1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:25.961044 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ea57f21a-221c-4a77-8c4d-d10909432547 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:25.962266 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ea57f21a-221c-4a77-8c4d-d10909432547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:25.967838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c3844-16df-4d04-a10b-2514ffd4bf1c admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:25.970174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1313/2636] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:25.972978 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba22dc-42d4-4a76-bac2-80df68606dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:25.973726 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba22dc-42d4-4a76-bac2-80df68606dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:25.974279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57f21a-221c-4a77-8c4d-d10909432547 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:25.974604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba22dc-42d4-4a76-bac2-80df68606dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:25.974882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba22dc-42d4-4a76-bac2-80df68606dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:25.975521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba22dc-42d4-4a76-bac2-80df68606dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:25.977175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba22dc-42d4-4a76-bac2-80df68606dc9 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:25.983395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57f21a-221c-4a77-8c4d-d10909432547 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:25.987955 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03655257-a3a7-4ddc-81ed-755fd1b19144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:25.988240 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1324/2637] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:25.988554 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03655257-a3a7-4ddc-81ed-755fd1b19144 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:25.988826 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03655257-a3a7-4ddc-81ed-755fd1b19144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:25.989094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03655257-a3a7-4ddc-81ed-755fd1b19144 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:25.989771 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03655257-a3a7-4ddc-81ed-755fd1b19144 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:56:25.990074 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03655257-a3a7-4ddc-81ed-755fd1b19144 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:25.991695 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03655257-a3a7-4ddc-81ed-755fd1b19144 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:25.992382 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba22dc-42d4-4a76-bac2-80df68606dc9 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:26.002411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba22dc-42d4-4a76-bac2-80df68606dc9 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:26.004426 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1314/2638] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:56:26.007048 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8017d06-6041-4192-99c2-7699d06ff86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:26.007642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8017d06-6041-4192-99c2-7699d06ff86e admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.007913 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8017d06-6041-4192-99c2-7699d06ff86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.008180 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8017d06-6041-4192-99c2-7699d06ff86e admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.008671 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03655257-a3a7-4ddc-81ed-755fd1b19144 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:26.008820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8017d06-6041-4192-99c2-7699d06ff86e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:26.010414 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8017d06-6041-4192-99c2-7699d06ff86e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:26.018669 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03655257-a3a7-4ddc-81ed-755fd1b19144 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:26.021333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1325/2639] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:26.023595 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6b1a1-f90d-4842-a838-809ea5540430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:26.024187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6b1a1-f90d-4842-a838-809ea5540430 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.024646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6b1a1-f90d-4842-a838-809ea5540430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.024773 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6b1a1-f90d-4842-a838-809ea5540430 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.025367 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6b1a1-f90d-4842-a838-809ea5540430 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:26.025682 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8017d06-6041-4192-99c2-7699d06ff86e admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:26.027319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6b1a1-f90d-4842-a838-809ea5540430 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:26.035687 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8017d06-6041-4192-99c2-7699d06ff86e admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:26.040803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac123eb-b25e-40de-b9de-e33236dc282c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:26.041666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac123eb-b25e-40de-b9de-e33236dc282c admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.041944 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac123eb-b25e-40de-b9de-e33236dc282c admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.042366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac123eb-b25e-40de-b9de-e33236dc282c admin admin] PATH_INFO: `/v3/projects/743d25d2ffa74853b6d0143873717fc2/users/7c24060271e14129af1b3afcf952e92e/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.049949 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:26.057987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac123eb-b25e-40de-b9de-e33236dc282c admin admin] RBAC: Authorizing `identity:create_grant(project_id=743d25d2ffa74853b6d0143873717fc2, user_id=7c24060271e14129af1b3afcf952e92e, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:26.061375 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1315/2640] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:26 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:26.061479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac123eb-b25e-40de-b9de-e33236dc282c admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:26.062339 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6b1a1-f90d-4842-a838-809ea5540430 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:26.074388 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1326/2641] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:26 2022] PUT /identity/v3/projects/743d25d2ffa74853b6d0143873717fc2/users/7c24060271e14129af1b3afcf952e92e/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:26.086052 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4b613-44de-4e45-b7ea-8049357aeab3 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.086366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4b613-44de-4e45-b7ea-8049357aeab3 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.086616 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4b613-44de-4e45-b7ea-8049357aeab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.116293 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-c2f4b613-44de-4e45-b7ea-8049357aeab3 None None] MFA Rules not processed for user `7c24060271e14129af1b3afcf952e92e`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:26.118356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c2f4b613-44de-4e45-b7ea-8049357aeab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:26.131738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6b1a1-f90d-4842-a838-809ea5540430 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:26.184574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1327/2642] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:26.188314 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc868f-ceae-402e-8c40-aa46e033536f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:26.188863 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4b613-44de-4e45-b7ea-8049357aeab3 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:26.188974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc868f-ceae-402e-8c40-aa46e033536f admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.189227 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc868f-ceae-402e-8c40-aa46e033536f admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.189498 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc868f-ceae-402e-8c40-aa46e033536f admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.190232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc868f-ceae-402e-8c40-aa46e033536f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:26.192199 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc868f-ceae-402e-8c40-aa46e033536f admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:26.199450 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4b613-44de-4e45-b7ea-8049357aeab3 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:26.206056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedb23e-1a12-4af4-aeb9-2d3a16188f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:26.206824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedb23e-1a12-4af4-aeb9-2d3a16188f17 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.207150 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedb23e-1a12-4af4-aeb9-2d3a16188f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.207462 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedb23e-1a12-4af4-aeb9-2d3a16188f17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.208213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedb23e-1a12-4af4-aeb9-2d3a16188f17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:26.210327 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedb23e-1a12-4af4-aeb9-2d3a16188f17 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:26.216895 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1316/2643] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:26 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:26.220685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc868f-ceae-402e-8c40-aa46e033536f admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:26.231274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc868f-ceae-402e-8c40-aa46e033536f admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:26.237806 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625a648-9b0a-4522-9f5a-9491187acfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:26.238444 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625a648-9b0a-4522-9f5a-9491187acfcf admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.238689 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625a648-9b0a-4522-9f5a-9491187acfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.238959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625a648-9b0a-4522-9f5a-9491187acfcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.239609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625a648-9b0a-4522-9f5a-9491187acfcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:26.241242 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625a648-9b0a-4522-9f5a-9491187acfcf admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:26.242372 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1328/2644] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:26 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:26.246459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedb23e-1a12-4af4-aeb9-2d3a16188f17 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:26.255292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1317/2645] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:26.256516 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625a648-9b0a-4522-9f5a-9491187acfcf admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:26.266214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625a648-9b0a-4522-9f5a-9491187acfcf admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:26.307003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedb23e-1a12-4af4-aeb9-2d3a16188f17 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:26.311729 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1a871-f160-4e9d-85ed-fa461d959895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:26.312336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1a871-f160-4e9d-85ed-fa461d959895 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.312605 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1a871-f160-4e9d-85ed-fa461d959895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.312869 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1a871-f160-4e9d-85ed-fa461d959895 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.313522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1a871-f160-4e9d-85ed-fa461d959895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:26.315447 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1a871-f160-4e9d-85ed-fa461d959895 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:26.323118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c978ae-d8fd-42cc-83ca-f96c51496c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:26.323906 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c978ae-d8fd-42cc-83ca-f96c51496c97 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.324703 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c978ae-d8fd-42cc-83ca-f96c51496c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.324987 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c978ae-d8fd-42cc-83ca-f96c51496c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.326079 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c978ae-d8fd-42cc-83ca-f96c51496c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:26.327682 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c978ae-d8fd-42cc-83ca-f96c51496c97 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:26.337377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1329/2646] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:26 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:26.340985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1318/2647] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:26.341941 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1a871-f160-4e9d-85ed-fa461d959895 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:26.346894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c978ae-d8fd-42cc-83ca-f96c51496c97 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:26.350549 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:26.355626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c978ae-d8fd-42cc-83ca-f96c51496c97 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:26.356879 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1a871-f160-4e9d-85ed-fa461d959895 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:26.360862 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d65be4-203d-45de-96c2-a0db5f931f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:26.361622 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d65be4-203d-45de-96c2-a0db5f931f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.361896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d65be4-203d-45de-96c2-a0db5f931f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.362160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d65be4-203d-45de-96c2-a0db5f931f33 admin admin] PATH_INFO: `/v3/projects/fa25b13404c54cff94e3c12198e9c313/users/00928159647d47d6b324c0c11231465e/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.363270 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeefdfb-72a8-4fa0-ad8c-74cc351541b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:26.363951 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeefdfb-72a8-4fa0-ad8c-74cc351541b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.364314 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeefdfb-72a8-4fa0-ad8c-74cc351541b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.364717 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeefdfb-72a8-4fa0-ad8c-74cc351541b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.365409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeefdfb-72a8-4fa0-ad8c-74cc351541b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:26.367018 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeefdfb-72a8-4fa0-ad8c-74cc351541b0 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:26.377834 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1330/2648] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:26.378026 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d65be4-203d-45de-96c2-a0db5f931f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa25b13404c54cff94e3c12198e9c313, user_id=00928159647d47d6b324c0c11231465e, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:26.380491 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d65be4-203d-45de-96c2-a0db5f931f33 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:26.382142 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeefdfb-72a8-4fa0-ad8c-74cc351541b0 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:26.383670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:26.392287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeefdfb-72a8-4fa0-ad8c-74cc351541b0 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:26.395945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1319/2649] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:26 2022] PUT /identity/v3/projects/fa25b13404c54cff94e3c12198e9c313/users/00928159647d47d6b324c0c11231465e/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:26.401029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9832b3f-d714-4599-9bac-e76cb29d441a None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.402395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9832b3f-d714-4599-9bac-e76cb29d441a None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.402937 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9832b3f-d714-4599-9bac-e76cb29d441a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.421954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-e9832b3f-d714-4599-9bac-e76cb29d441a None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:26.423210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e9832b3f-d714-4599-9bac-e76cb29d441a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:26.450064 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f891b7-19a0-47aa-971d-5b686cb15025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:26.450674 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f891b7-19a0-47aa-971d-5b686cb15025 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.450955 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f891b7-19a0-47aa-971d-5b686cb15025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.451222 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f891b7-19a0-47aa-971d-5b686cb15025 admin admin] PATH_INFO: `/v3/projects/702737d8412743d48e8da66462e91164/users/ca27c09b92f848199553dd8303415138/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.466995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f891b7-19a0-47aa-971d-5b686cb15025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=702737d8412743d48e8da66462e91164, user_id=ca27c09b92f848199553dd8303415138, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:26.470303 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f891b7-19a0-47aa-971d-5b686cb15025 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:26.483869 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1331/2650] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:26 2022] PUT /identity/v3/projects/702737d8412743d48e8da66462e91164/users/ca27c09b92f848199553dd8303415138/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:26.487577 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1320/2651] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:26.488196 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f891b7-19a0-47aa-971d-5b686cb15025 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:26.498446 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9832b3f-d714-4599-9bac-e76cb29d441a None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:26.500765 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e9832b3f-d714-4599-9bac-e76cb29d441a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:26.552942 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f891b7-19a0-47aa-971d-5b686cb15025 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:26.558669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579562c2-38d0-4889-91f9-b584337c980f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:26.559327 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579562c2-38d0-4889-91f9-b584337c980f admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.559655 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579562c2-38d0-4889-91f9-b584337c980f admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.559932 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579562c2-38d0-4889-91f9-b584337c980f admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.560575 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579562c2-38d0-4889-91f9-b584337c980f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:26.562205 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579562c2-38d0-4889-91f9-b584337c980f admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:26.569160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9832b3f-d714-4599-9bac-e76cb29d441a None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:26.572658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1332/2652] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:26.576923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579562c2-38d0-4889-91f9-b584337c980f admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:26.588969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579562c2-38d0-4889-91f9-b584337c980f admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:26.595022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bc011-cfd1-47d9-9a9f-02569e6e0a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:26.595899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bc011-cfd1-47d9-9a9f-02569e6e0a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.596201 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bc011-cfd1-47d9-9a9f-02569e6e0a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.596509 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bc011-cfd1-47d9-9a9f-02569e6e0a20 admin admin] PATH_INFO: `/v3/projects/702737d8412743d48e8da66462e91164/users/ca27c09b92f848199553dd8303415138/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.598629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bc011-cfd1-47d9-9a9f-02569e6e0a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=702737d8412743d48e8da66462e91164, user_id=ca27c09b92f848199553dd8303415138, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:26.601400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bc011-cfd1-47d9-9a9f-02569e6e0a20 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:26.612747 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1333/2653] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:26 2022] PUT /identity/v3/projects/702737d8412743d48e8da66462e91164/users/ca27c09b92f848199553dd8303415138/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:26.622620 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968284c0-c5cb-4442-bfa7-980025e07e1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:26.623252 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968284c0-c5cb-4442-bfa7-980025e07e1f service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.623533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968284c0-c5cb-4442-bfa7-980025e07e1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.623811 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968284c0-c5cb-4442-bfa7-980025e07e1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.624728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0a367-8b05-4aa6-951b-24d228b8cfec None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.625013 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0a367-8b05-4aa6-951b-24d228b8cfec None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.625322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0a367-8b05-4aa6-951b-24d228b8cfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.633848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968284c0-c5cb-4442-bfa7-980025e07e1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:26.635300 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968284c0-c5cb-4442-bfa7-980025e07e1f service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:26.645871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1321/2654] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:56:26.658407 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-4ea0a367-8b05-4aa6-951b-24d228b8cfec None None] MFA Rules not processed for user `ca27c09b92f848199553dd8303415138`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:26.659633 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4ea0a367-8b05-4aa6-951b-24d228b8cfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:26.717757 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:26.745065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1334/2655] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:26.759617 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968284c0-c5cb-4442-bfa7-980025e07e1f service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:26.761093 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-968284c0-c5cb-4442-bfa7-980025e07e1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:26.820739 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968284c0-c5cb-4442-bfa7-980025e07e1f service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:26.829342 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=100 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:26.876491 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6daace-d5f4-4ea0-b5a5-f1fbb0a116e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:26.877178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6daace-d5f4-4ea0-b5a5-f1fbb0a116e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:26.877421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6daace-d5f4-4ea0-b5a5-f1fbb0a116e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:26.877670 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6daace-d5f4-4ea0-b5a5-f1fbb0a116e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:26.887537 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6daace-d5f4-4ea0-b5a5-f1fbb0a116e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:26.888972 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6daace-d5f4-4ea0-b5a5-f1fbb0a116e0 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:26.900367 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1322/2656] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:27.390123 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661c977-938b-442c-a7b0-c0b2b4aff254 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:27.390645 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661c977-938b-442c-a7b0-c0b2b4aff254 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:27.390776 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661c977-938b-442c-a7b0-c0b2b4aff254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:27.448078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-5661c977-938b-442c-a7b0-c0b2b4aff254 None None] MFA Rules not processed for user `00928159647d47d6b324c0c11231465e`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:27.449678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5661c977-938b-442c-a7b0-c0b2b4aff254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:27.507667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6daace-d5f4-4ea0-b5a5-f1fbb0a116e0 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:27.546658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1335/2657] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:27.557943 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661c977-938b-442c-a7b0-c0b2b4aff254 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:27.568703 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661c977-938b-442c-a7b0-c0b2b4aff254 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:27.573408 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85c4b3-fc2c-40c5-a901-c392f5bb7747 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:27.574215 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85c4b3-fc2c-40c5-a901-c392f5bb7747 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:27.574613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85c4b3-fc2c-40c5-a901-c392f5bb7747 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:27.575422 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85c4b3-fc2c-40c5-a901-c392f5bb7747 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:27.581017 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6daace-d5f4-4ea0-b5a5-f1fbb0a116e0 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:27.585204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85c4b3-fc2c-40c5-a901-c392f5bb7747 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:27.586675 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85c4b3-fc2c-40c5-a901-c392f5bb7747 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:27.586940 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822be712-4271-4ef9-93b5-f9c75882f439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:27.587651 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822be712-4271-4ef9-93b5-f9c75882f439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:27.587995 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822be712-4271-4ef9-93b5-f9c75882f439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:27.588331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822be712-4271-4ef9-93b5-f9c75882f439 admin admin] PATH_INFO: `/v3/users/7c24060271e14129af1b3afcf952e92e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:27.589614 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822be712-4271-4ef9-93b5-f9c75882f439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c24060271e14129af1b3afcf952e92e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:27.591688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822be712-4271-4ef9-93b5-f9c75882f439 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:27.597615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1336/2658] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:56:27.643836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-822be712-4271-4ef9-93b5-f9c75882f439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c24060271e14129af1b3afcf952e92e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:27.652135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-822be712-4271-4ef9-93b5-f9c75882f439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c24060271e14129af1b3afcf952e92e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:27.652778 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-822be712-4271-4ef9-93b5-f9c75882f439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c24060271e14129af1b3afcf952e92e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:27.653398 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-822be712-4271-4ef9-93b5-f9c75882f439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c24060271e14129af1b3afcf952e92e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:27.666856 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-822be712-4271-4ef9-93b5-f9c75882f439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c24060271e14129af1b3afcf952e92e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:27.679504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1323/2659] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:27 2022] DELETE /identity/v3/users/7c24060271e14129af1b3afcf952e92e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:27.683878 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459c791-0b5c-4994-9675-cd1fb8743436 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:27.684336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459c791-0b5c-4994-9675-cd1fb8743436 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:27.684717 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459c791-0b5c-4994-9675-cd1fb8743436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:27.704444 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-9459c791-0b5c-4994-9675-cd1fb8743436 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:27.706142 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9459c791-0b5c-4994-9675-cd1fb8743436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:27.779336 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1337/2660] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:27.790065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822be712-4271-4ef9-93b5-f9c75882f439 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:27.791515 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-822be712-4271-4ef9-93b5-f9c75882f439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:27.832312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:27.858444 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822be712-4271-4ef9-93b5-f9c75882f439 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:27.862612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee93412-d767-4309-b211-c1c1e281fed6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:27.863283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee93412-d767-4309-b211-c1c1e281fed6 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:27.863565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee93412-d767-4309-b211-c1c1e281fed6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:27.863832 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee93412-d767-4309-b211-c1c1e281fed6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:27.874619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee93412-d767-4309-b211-c1c1e281fed6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:27.876082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee93412-d767-4309-b211-c1c1e281fed6 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:27.886454 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1324/2661] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:28.176198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459c791-0b5c-4994-9675-cd1fb8743436 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:28.177790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9459c791-0b5c-4994-9675-cd1fb8743436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:28.190145 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459c791-0b5c-4994-9675-cd1fb8743436 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:28.195843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523fef3-c0cf-438c-af2e-e664197d9191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:28.196614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523fef3-c0cf-438c-af2e-e664197d9191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:28.196893 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523fef3-c0cf-438c-af2e-e664197d9191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:28.197160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523fef3-c0cf-438c-af2e-e664197d9191 admin admin] PATH_INFO: `/v3/projects/743d25d2ffa74853b6d0143873717fc2` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:28.198451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523fef3-c0cf-438c-af2e-e664197d9191 admin admin] RBAC: Authorizing `identity:delete_project(project_id=743d25d2ffa74853b6d0143873717fc2)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:28.200547 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523fef3-c0cf-438c-af2e-e664197d9191 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:28.254669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9523fef3-c0cf-438c-af2e-e664197d9191 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '743d25d2ffa74853b6d0143873717fc2', 'request_id': 'req-9523fef3-c0cf-438c-af2e-e664197d9191'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:28.261144 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9523fef3-c0cf-438c-af2e-e664197d9191 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '743d25d2ffa74853b6d0143873717fc2', 'request_id': 'req-9523fef3-c0cf-438c-af2e-e664197d9191'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:28.263838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:28.270657 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9523fef3-c0cf-438c-af2e-e664197d9191 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '743d25d2ffa74853b6d0143873717fc2', 'request_id': 'req-9523fef3-c0cf-438c-af2e-e664197d9191'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:28.282550 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9523fef3-c0cf-438c-af2e-e664197d9191 admin admin] The token cache is being invalidate because project 743d25d2ffa74853b6d0143873717fc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:28.282828 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9523fef3-c0cf-438c-af2e-e664197d9191 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:28.283376 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9523fef3-c0cf-438c-af2e-e664197d9191 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:28.285801 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1338/2662] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:28 2022] DELETE /identity/v3/projects/743d25d2ffa74853b6d0143873717fc2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:28.291071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee93412-d767-4309-b211-c1c1e281fed6 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:28.301566 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee93412-d767-4309-b211-c1c1e281fed6 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:28.366009 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad117461-9567-40bf-b4e4-668e48852826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:28.366655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad117461-9567-40bf-b4e4-668e48852826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:28.366934 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad117461-9567-40bf-b4e4-668e48852826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:28.367196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad117461-9567-40bf-b4e4-668e48852826 admin admin] PATH_INFO: `/v3/users/ca27c09b92f848199553dd8303415138` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:28.368573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad117461-9567-40bf-b4e4-668e48852826 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca27c09b92f848199553dd8303415138)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:28.370576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad117461-9567-40bf-b4e4-668e48852826 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:28.427824 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ad117461-9567-40bf-b4e4-668e48852826 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca27c09b92f848199553dd8303415138'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:28.436517 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ad117461-9567-40bf-b4e4-668e48852826 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca27c09b92f848199553dd8303415138'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:28.437185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ad117461-9567-40bf-b4e4-668e48852826 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca27c09b92f848199553dd8303415138'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:28.437810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ad117461-9567-40bf-b4e4-668e48852826 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca27c09b92f848199553dd8303415138'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:28.450620 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ad117461-9567-40bf-b4e4-668e48852826 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca27c09b92f848199553dd8303415138'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:28.463298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1325/2663] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:28 2022] DELETE /identity/v3/users/ca27c09b92f848199553dd8303415138 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:28.750035 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523fef3-c0cf-438c-af2e-e664197d9191 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:28.751499 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9523fef3-c0cf-438c-af2e-e664197d9191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:28.764846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523fef3-c0cf-438c-af2e-e664197d9191 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:28.823815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f3af5-a0f3-4f6b-8ba0-69e52c120b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:28.824451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f3af5-a0f3-4f6b-8ba0-69e52c120b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:28.824736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f3af5-a0f3-4f6b-8ba0-69e52c120b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:28.825010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f3af5-a0f3-4f6b-8ba0-69e52c120b42 admin admin] PATH_INFO: `/v3/projects/702737d8412743d48e8da66462e91164` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:28.826178 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f3af5-a0f3-4f6b-8ba0-69e52c120b42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=702737d8412743d48e8da66462e91164)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:28.828072 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f3af5-a0f3-4f6b-8ba0-69e52c120b42 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:28.877122 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-337f3af5-a0f3-4f6b-8ba0-69e52c120b42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '702737d8412743d48e8da66462e91164', 'request_id': 'req-337f3af5-a0f3-4f6b-8ba0-69e52c120b42'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:28.883372 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-337f3af5-a0f3-4f6b-8ba0-69e52c120b42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '702737d8412743d48e8da66462e91164', 'request_id': 'req-337f3af5-a0f3-4f6b-8ba0-69e52c120b42'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:28.892901 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-337f3af5-a0f3-4f6b-8ba0-69e52c120b42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '702737d8412743d48e8da66462e91164', 'request_id': 'req-337f3af5-a0f3-4f6b-8ba0-69e52c120b42'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:28.905349 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-337f3af5-a0f3-4f6b-8ba0-69e52c120b42 admin admin] The token cache is being invalidate because project 702737d8412743d48e8da66462e91164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:28.905628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-337f3af5-a0f3-4f6b-8ba0-69e52c120b42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:28.906282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-337f3af5-a0f3-4f6b-8ba0-69e52c120b42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:28.907769 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1339/2664] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:28 2022] DELETE /identity/v3/projects/702737d8412743d48e8da66462e91164 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:28.912785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4392f7b-edcb-4fb3-be9c-24010c6c19c9 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:28.913346 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4392f7b-edcb-4fb3-be9c-24010c6c19c9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:28.913346 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4392f7b-edcb-4fb3-be9c-24010c6c19c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:28.933587 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c4392f7b-edcb-4fb3-be9c-24010c6c19c9 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:28.934797 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c4392f7b-edcb-4fb3-be9c-24010c6c19c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:28.987561 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:29.003530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1326/2665] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:29.012083 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c031834-4ee4-4143-96c4-893f0463d97d None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:29.012352 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c031834-4ee4-4143-96c4-893f0463d97d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:29.012637 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c031834-4ee4-4143-96c4-893f0463d97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:29.036481 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-6c031834-4ee4-4143-96c4-893f0463d97d None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:29.037856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-6c031834-4ee4-4143-96c4-893f0463d97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:29.046532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1340/2666] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:29.048827 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3ca38-d82f-4a43-a25e-2728aef2b188 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:29.049109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3ca38-d82f-4a43-a25e-2728aef2b188 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:29.049410 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3ca38-d82f-4a43-a25e-2728aef2b188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:29.050692 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c031834-4ee4-4143-96c4-893f0463d97d None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:29.060976 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c031834-4ee4-4143-96c4-893f0463d97d None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:29.066028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373ab4d-0d09-41f1-8c4e-43131f815c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:29.066623 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373ab4d-0d09-41f1-8c4e-43131f815c86 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:29.066895 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373ab4d-0d09-41f1-8c4e-43131f815c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:29.067163 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373ab4d-0d09-41f1-8c4e-43131f815c86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:29.067796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373ab4d-0d09-41f1-8c4e-43131f815c86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:56:29.068090 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373ab4d-0d09-41f1-8c4e-43131f815c86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:29.068671 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-21f3ca38-d82f-4a43-a25e-2728aef2b188 None None] MFA Rules not processed for user `00928159647d47d6b324c0c11231465e`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:29.069709 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373ab4d-0d09-41f1-8c4e-43131f815c86 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:29.069921 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-21f3ca38-d82f-4a43-a25e-2728aef2b188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:29.085860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1341/2667] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:56:29.087029 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373ab4d-0d09-41f1-8c4e-43131f815c86 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:29.097204 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373ab4d-0d09-41f1-8c4e-43131f815c86 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:29.103211 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe05c2c-d7db-4c6f-81fe-93bf27c485c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:29.104079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe05c2c-d7db-4c6f-81fe-93bf27c485c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:29.104479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe05c2c-d7db-4c6f-81fe-93bf27c485c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:29.104856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe05c2c-d7db-4c6f-81fe-93bf27c485c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:29.105562 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe05c2c-d7db-4c6f-81fe-93bf27c485c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:29.107471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe05c2c-d7db-4c6f-81fe-93bf27c485c6 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:29.132795 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1327/2668] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:29.136237 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1342/2669] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:29 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:29.141259 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3ca38-d82f-4a43-a25e-2728aef2b188 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:29.152805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3ca38-d82f-4a43-a25e-2728aef2b188 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:29.216879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ce486-f5a6-4c84-86d2-37085e1691e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:29.218531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ce486-f5a6-4c84-86d2-37085e1691e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:29.218812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ce486-f5a6-4c84-86d2-37085e1691e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:29.219091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ce486-f5a6-4c84-86d2-37085e1691e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:29.219874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ce486-f5a6-4c84-86d2-37085e1691e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:29.221841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ce486-f5a6-4c84-86d2-37085e1691e7 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:29.246445 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1328/2670] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:29 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:29.250892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe05c2c-d7db-4c6f-81fe-93bf27c485c6 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:29.261110 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe05c2c-d7db-4c6f-81fe-93bf27c485c6 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:29.265735 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1dcc27-5be3-4958-a0ba-f2ef13944d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:29.266337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dcc27-5be3-4958-a0ba-f2ef13944d5d admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:29.266613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dcc27-5be3-4958-a0ba-f2ef13944d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:29.266876 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dcc27-5be3-4958-a0ba-f2ef13944d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:29.267529 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1dcc27-5be3-4958-a0ba-f2ef13944d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:29.269121 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1dcc27-5be3-4958-a0ba-f2ef13944d5d admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:29.279361 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1343/2671] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:29.283366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ce486-f5a6-4c84-86d2-37085e1691e7 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:29.293968 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ce486-f5a6-4c84-86d2-37085e1691e7 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:29.299260 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac3ec2-b6de-4cd1-b7d9-adf548b5a593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:29.299873 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac3ec2-b6de-4cd1-b7d9-adf548b5a593 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:29.300147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac3ec2-b6de-4cd1-b7d9-adf548b5a593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:29.300419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac3ec2-b6de-4cd1-b7d9-adf548b5a593 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:29.301071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac3ec2-b6de-4cd1-b7d9-adf548b5a593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:29.302718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac3ec2-b6de-4cd1-b7d9-adf548b5a593 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:29.313415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1329/2672] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:29.317715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1dcc27-5be3-4958-a0ba-f2ef13944d5d admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:29.327852 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1dcc27-5be3-4958-a0ba-f2ef13944d5d admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:29.333415 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c3231-47cb-48e5-ad48-4f20faa7ccb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:29.334070 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c3231-47cb-48e5-ad48-4f20faa7ccb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:29.334348 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c3231-47cb-48e5-ad48-4f20faa7ccb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:29.334618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c3231-47cb-48e5-ad48-4f20faa7ccb6 admin admin] PATH_INFO: `/v3/projects/edce948f32c641f9bd25fc38ec1742be/users/2298ff79dfb744a381034545870ca220/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:29.350526 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c3231-47cb-48e5-ad48-4f20faa7ccb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edce948f32c641f9bd25fc38ec1742be, user_id=2298ff79dfb744a381034545870ca220, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:29.353194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c3231-47cb-48e5-ad48-4f20faa7ccb6 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:29.362439 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:29.368316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1344/2673] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:29 2022] PUT /identity/v3/projects/edce948f32c641f9bd25fc38ec1742be/users/2298ff79dfb744a381034545870ca220/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:29.378483 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac3ec2-b6de-4cd1-b7d9-adf548b5a593 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:29.380492 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-74ac3ec2-b6de-4cd1-b7d9-adf548b5a593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:29.380899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363893e0-7809-43ad-b0e8-38383359c031 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:29.381217 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363893e0-7809-43ad-b0e8-38383359c031 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:29.381496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363893e0-7809-43ad-b0e8-38383359c031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:29.410409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-363893e0-7809-43ad-b0e8-38383359c031 None None] MFA Rules not processed for user `2298ff79dfb744a381034545870ca220`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:29.411626 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-363893e0-7809-43ad-b0e8-38383359c031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:29.442628 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac3ec2-b6de-4cd1-b7d9-adf548b5a593 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:29.477400 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:29.479191 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1345/2674] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:29.483748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363893e0-7809-43ad-b0e8-38383359c031 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:29.496198 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3a46f-bc4f-4ad7-b629-af7fc88ac830 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:29.496813 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3a46f-bc4f-4ad7-b629-af7fc88ac830 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:29.497094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3a46f-bc4f-4ad7-b629-af7fc88ac830 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:29.497415 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3a46f-bc4f-4ad7-b629-af7fc88ac830 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:29.499958 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-94d3a46f-bc4f-4ad7-b629-af7fc88ac830 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:29.509656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3a46f-bc4f-4ad7-b629-af7fc88ac830 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:29.511101 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3a46f-bc4f-4ad7-b629-af7fc88ac830 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:29.519755 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1330/2675] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:29.550389 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363893e0-7809-43ad-b0e8-38383359c031 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:29.555393 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc63f81-48f3-4a5d-b14b-784fd0fc30cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:29.556277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc63f81-48f3-4a5d-b14b-784fd0fc30cf admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:29.556554 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc63f81-48f3-4a5d-b14b-784fd0fc30cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:29.556830 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc63f81-48f3-4a5d-b14b-784fd0fc30cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:29.557509 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc63f81-48f3-4a5d-b14b-784fd0fc30cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:29.559615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc63f81-48f3-4a5d-b14b-784fd0fc30cf admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:29.591407 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1346/2676] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:29 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:29.592888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3a46f-bc4f-4ad7-b629-af7fc88ac830 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:29.605486 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3a46f-bc4f-4ad7-b629-af7fc88ac830 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:29.670431 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea121b-43e8-4e07-83ee-b9f15071e982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:29.670848 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea121b-43e8-4e07-83ee-b9f15071e982 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:29.671130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea121b-43e8-4e07-83ee-b9f15071e982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:29.673543 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea121b-43e8-4e07-83ee-b9f15071e982 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:29.675644 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea121b-43e8-4e07-83ee-b9f15071e982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:29.678219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea121b-43e8-4e07-83ee-b9f15071e982 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:29.721942 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1331/2677] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:29 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:29.726144 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc63f81-48f3-4a5d-b14b-784fd0fc30cf admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:29.737837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc63f81-48f3-4a5d-b14b-784fd0fc30cf admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:29.743000 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac752515-22e3-45e7-a40f-a22c43b7c92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:29.743610 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac752515-22e3-45e7-a40f-a22c43b7c92e admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:29.743882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac752515-22e3-45e7-a40f-a22c43b7c92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:29.744148 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac752515-22e3-45e7-a40f-a22c43b7c92e admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:29.744796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac752515-22e3-45e7-a40f-a22c43b7c92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:29.746442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac752515-22e3-45e7-a40f-a22c43b7c92e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:29.761224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1347/2678] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:29.763351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea121b-43e8-4e07-83ee-b9f15071e982 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:29.773826 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea121b-43e8-4e07-83ee-b9f15071e982 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:29.778721 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564dd59-399f-4a2d-bb70-97b769953276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:29.779399 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564dd59-399f-4a2d-bb70-97b769953276 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:29.779672 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564dd59-399f-4a2d-bb70-97b769953276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:29.779963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564dd59-399f-4a2d-bb70-97b769953276 admin admin] PATH_INFO: `/v3/projects/560ff79b6fcd462bb236a45953bedbc1/users/a7f8885301d64b82add03032cdb0930c/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:29.799703 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564dd59-399f-4a2d-bb70-97b769953276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=560ff79b6fcd462bb236a45953bedbc1, user_id=a7f8885301d64b82add03032cdb0930c, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:29.802531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564dd59-399f-4a2d-bb70-97b769953276 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:29.822624 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1332/2679] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:29 2022] PUT /identity/v3/projects/560ff79b6fcd462bb236a45953bedbc1/users/a7f8885301d64b82add03032cdb0930c/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:29.824503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac752515-22e3-45e7-a40f-a22c43b7c92e admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:29.836933 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac752515-22e3-45e7-a40f-a22c43b7c92e admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:29.898247 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5613c1-6ad6-4c1b-83fb-b8cb2bcdb2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:29.898846 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5613c1-6ad6-4c1b-83fb-b8cb2bcdb2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:29.899120 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5613c1-6ad6-4c1b-83fb-b8cb2bcdb2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:29.899421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5613c1-6ad6-4c1b-83fb-b8cb2bcdb2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:29.900051 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5613c1-6ad6-4c1b-83fb-b8cb2bcdb2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:29.901687 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5613c1-6ad6-4c1b-83fb-b8cb2bcdb2c7 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:29.903860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:29.912510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1348/2680] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:29.916329 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564dd59-399f-4a2d-bb70-97b769953276 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:29.927492 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564dd59-399f-4a2d-bb70-97b769953276 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:29.932310 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fad1ac-30a7-40a7-b5c8-18e49dd608cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:29.932985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fad1ac-30a7-40a7-b5c8-18e49dd608cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:29.933305 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fad1ac-30a7-40a7-b5c8-18e49dd608cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:29.933580 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fad1ac-30a7-40a7-b5c8-18e49dd608cf admin admin] PATH_INFO: `/v3/projects/560ff79b6fcd462bb236a45953bedbc1/users/a7f8885301d64b82add03032cdb0930c/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:29.935557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fad1ac-30a7-40a7-b5c8-18e49dd608cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=560ff79b6fcd462bb236a45953bedbc1, user_id=a7f8885301d64b82add03032cdb0930c, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:29.940688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fad1ac-30a7-40a7-b5c8-18e49dd608cf admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:29.949992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1333/2681] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:29 2022] PUT /identity/v3/projects/560ff79b6fcd462bb236a45953bedbc1/users/a7f8885301d64b82add03032cdb0930c/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:29.961918 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ac431-7edb-4d58-9987-f6588f8cc5d5 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:29.962203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ac431-7edb-4d58-9987-f6588f8cc5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:29.962514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ac431-7edb-4d58-9987-f6588f8cc5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:29.996299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-071ac431-7edb-4d58-9987-f6588f8cc5d5 None None] MFA Rules not processed for user `a7f8885301d64b82add03032cdb0930c`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:29.997658 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-071ac431-7edb-4d58-9987-f6588f8cc5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:30.068002 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1349/2682] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:30.079985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fad1ac-30a7-40a7-b5c8-18e49dd608cf admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:30.081954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-90fad1ac-30a7-40a7-b5c8-18e49dd608cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:30.143781 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fad1ac-30a7-40a7-b5c8-18e49dd608cf admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:30.148527 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5998a6e-1736-425e-ae6f-9231aa4ca33f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:30.149294 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5998a6e-1736-425e-ae6f-9231aa4ca33f service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:30.149575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5998a6e-1736-425e-ae6f-9231aa4ca33f service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:30.149846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5998a6e-1736-425e-ae6f-9231aa4ca33f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:30.159831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5998a6e-1736-425e-ae6f-9231aa4ca33f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:30.161293 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5998a6e-1736-425e-ae6f-9231aa4ca33f service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:30.171204 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1334/2683] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:30 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:30.263252 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ac431-7edb-4d58-9987-f6588f8cc5d5 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:30.274593 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ac431-7edb-4d58-9987-f6588f8cc5d5 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:30.278851 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f120df-5c04-4d0a-ab0d-b8aa35f2a081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:30.279455 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f120df-5c04-4d0a-ab0d-b8aa35f2a081 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:30.279745 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f120df-5c04-4d0a-ab0d-b8aa35f2a081 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:30.280010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f120df-5c04-4d0a-ab0d-b8aa35f2a081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:30.341079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f120df-5c04-4d0a-ab0d-b8aa35f2a081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:30.342445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f120df-5c04-4d0a-ab0d-b8aa35f2a081 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:30.352995 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1350/2684] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:30.897780 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5998a6e-1736-425e-ae6f-9231aa4ca33f service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:30.899392 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d5998a6e-1736-425e-ae6f-9231aa4ca33f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:30.937888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:30.975555 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5998a6e-1736-425e-ae6f-9231aa4ca33f service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:30.981166 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694a946-3e13-47f4-8387-c299d1abeabe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:30.981787 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694a946-3e13-47f4-8387-c299d1abeabe service placement] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:30.982066 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694a946-3e13-47f4-8387-c299d1abeabe service placement] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:30.982339 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694a946-3e13-47f4-8387-c299d1abeabe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:30.984234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6694a946-3e13-47f4-8387-c299d1abeabe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:30.997046 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694a946-3e13-47f4-8387-c299d1abeabe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:30.998661 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694a946-3e13-47f4-8387-c299d1abeabe service placement] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:31.009175 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1335/2685] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:32.820055 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f120df-5c04-4d0a-ab0d-b8aa35f2a081 service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:32.822377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d1f120df-5c04-4d0a-ab0d-b8aa35f2a081 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:32.896161 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f120df-5c04-4d0a-ab0d-b8aa35f2a081 service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:32.901490 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bda793-96ec-440b-9fa3-62357d34fe8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:32.902081 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bda793-96ec-440b-9fa3-62357d34fe8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:32.902377 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bda793-96ec-440b-9fa3-62357d34fe8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:32.902654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bda793-96ec-440b-9fa3-62357d34fe8d admin admin] PATH_INFO: `/v3/users/aaac001a38a74cee9eaeb49d6b5900f8` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:32.903794 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bda793-96ec-440b-9fa3-62357d34fe8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaac001a38a74cee9eaeb49d6b5900f8)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:32.905807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bda793-96ec-440b-9fa3-62357d34fe8d admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:32.928736 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:32.953926 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-33bda793-96ec-440b-9fa3-62357d34fe8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaac001a38a74cee9eaeb49d6b5900f8'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:32.962901 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-33bda793-96ec-440b-9fa3-62357d34fe8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaac001a38a74cee9eaeb49d6b5900f8'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:32.963516 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-33bda793-96ec-440b-9fa3-62357d34fe8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaac001a38a74cee9eaeb49d6b5900f8'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:32.964175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-33bda793-96ec-440b-9fa3-62357d34fe8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaac001a38a74cee9eaeb49d6b5900f8'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:32.971702 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-33bda793-96ec-440b-9fa3-62357d34fe8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaac001a38a74cee9eaeb49d6b5900f8'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:32.990361 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1351/2686] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:32 2022] DELETE /identity/v3/users/aaac001a38a74cee9eaeb49d6b5900f8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:32.995137 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0a098-4e0e-4c6a-bced-6664d07de6ea None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:32.995729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0a098-4e0e-4c6a-bced-6664d07de6ea None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:32.995913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0a098-4e0e-4c6a-bced-6664d07de6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:33.014924 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-20d0a098-4e0e-4c6a-bced-6664d07de6ea None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:33.016237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-20d0a098-4e0e-4c6a-bced-6664d07de6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:33.086216 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1336/2687] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:33.095798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bda793-96ec-440b-9fa3-62357d34fe8d admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:33.097330 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-33bda793-96ec-440b-9fa3-62357d34fe8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:33.170549 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bda793-96ec-440b-9fa3-62357d34fe8d admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:33.174894 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ac60d-6939-48bb-865d-e338978010e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:33.175542 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ac60d-6939-48bb-865d-e338978010e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:33.175823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ac60d-6939-48bb-865d-e338978010e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:33.176412 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ac60d-6939-48bb-865d-e338978010e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:33.190898 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ac60d-6939-48bb-865d-e338978010e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:33.192369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ac60d-6939-48bb-865d-e338978010e2 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:33.199975 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d0a098-4e0e-4c6a-bced-6664d07de6ea None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:33.202133 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-20d0a098-4e0e-4c6a-bced-6664d07de6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:33.205559 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1352/2688] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:33.221355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d0a098-4e0e-4c6a-bced-6664d07de6ea None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:33.226860 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c9994-c2f9-4877-bc90-931cad679e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:33.227516 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c9994-c2f9-4877-bc90-931cad679e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:33.227796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c9994-c2f9-4877-bc90-931cad679e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:33.228075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c9994-c2f9-4877-bc90-931cad679e29 admin admin] PATH_INFO: `/v3/users/2298ff79dfb744a381034545870ca220` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:33.237247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c9994-c2f9-4877-bc90-931cad679e29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2298ff79dfb744a381034545870ca220)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:33.239219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c9994-c2f9-4877-bc90-931cad679e29 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:33.308853 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-015c9994-c2f9-4877-bc90-931cad679e29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2298ff79dfb744a381034545870ca220'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:33.332648 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-015c9994-c2f9-4877-bc90-931cad679e29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2298ff79dfb744a381034545870ca220'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:33.333330 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-015c9994-c2f9-4877-bc90-931cad679e29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2298ff79dfb744a381034545870ca220'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:33.334120 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-015c9994-c2f9-4877-bc90-931cad679e29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2298ff79dfb744a381034545870ca220'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:33.359581 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-015c9994-c2f9-4877-bc90-931cad679e29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2298ff79dfb744a381034545870ca220'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:33.377648 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1337/2689] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:33 2022] DELETE /identity/v3/users/2298ff79dfb744a381034545870ca220 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:33.383227 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0317e-46b2-4950-8bd5-4f406cddee58 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:33.383841 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0317e-46b2-4950-8bd5-4f406cddee58 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:33.384010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0317e-46b2-4950-8bd5-4f406cddee58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:33.428108 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-04b0317e-46b2-4950-8bd5-4f406cddee58 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:33.429850 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-04b0317e-46b2-4950-8bd5-4f406cddee58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:33.522378 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c9994-c2f9-4877-bc90-931cad679e29 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:33.524132 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-015c9994-c2f9-4877-bc90-931cad679e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:33.532912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1353/2690] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:33.547691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0317e-46b2-4950-8bd5-4f406cddee58 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:33.549906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-04b0317e-46b2-4950-8bd5-4f406cddee58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:33.561426 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c9994-c2f9-4877-bc90-931cad679e29 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:33.566211 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:33.566864 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:33.567148 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:33.567476 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83 admin admin] PATH_INFO: `/v3/projects/eb022978bd494438bf818c7543cd1733` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:33.568556 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb022978bd494438bf818c7543cd1733)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:33.570527 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:33.606827 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:33.611449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:33.624286 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0317e-46b2-4950-8bd5-4f406cddee58 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:33.631001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb022978bd494438bf818c7543cd1733', 'request_id': 'req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:33.638199 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb022978bd494438bf818c7543cd1733', 'request_id': 'req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:33.647554 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb022978bd494438bf818c7543cd1733', 'request_id': 'req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:33.663674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83 admin admin] The token cache is being invalidate because project eb022978bd494438bf818c7543cd1733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:33.664263 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:33.665449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:33.668581 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1338/2691] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:33 2022] DELETE /identity/v3/projects/eb022978bd494438bf818c7543cd1733 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:33.673179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:33.675107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:33.680026 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76110cd3-6e32-491b-804a-4cb12383be4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:33.680872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76110cd3-6e32-491b-804a-4cb12383be4c service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:33.681009 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76110cd3-6e32-491b-804a-4cb12383be4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:33.681234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76110cd3-6e32-491b-804a-4cb12383be4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:33.683022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-76110cd3-6e32-491b-804a-4cb12383be4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:33.773120 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a6be3-aa97-4a5a-b97b-d71ea38a8d83 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:33.773811 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76110cd3-6e32-491b-804a-4cb12383be4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:33.775804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76110cd3-6e32-491b-804a-4cb12383be4c service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:33.779094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500fa5c-b394-4e93-8a54-a988bafcfa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:33.779771 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500fa5c-b394-4e93-8a54-a988bafcfa0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:33.780047 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500fa5c-b394-4e93-8a54-a988bafcfa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:33.780318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500fa5c-b394-4e93-8a54-a988bafcfa0d admin admin] PATH_INFO: `/v3/users/da297351b65f4fa1b31bf660b1399bbd` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:33.781518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500fa5c-b394-4e93-8a54-a988bafcfa0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=da297351b65f4fa1b31bf660b1399bbd)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:33.783469 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500fa5c-b394-4e93-8a54-a988bafcfa0d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:33.791095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1354/2692] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:33.841910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5500fa5c-b394-4e93-8a54-a988bafcfa0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da297351b65f4fa1b31bf660b1399bbd'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:33.850738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5500fa5c-b394-4e93-8a54-a988bafcfa0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da297351b65f4fa1b31bf660b1399bbd'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:33.852513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5500fa5c-b394-4e93-8a54-a988bafcfa0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da297351b65f4fa1b31bf660b1399bbd'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:33.852814 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5500fa5c-b394-4e93-8a54-a988bafcfa0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da297351b65f4fa1b31bf660b1399bbd'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:33.870445 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-5500fa5c-b394-4e93-8a54-a988bafcfa0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da297351b65f4fa1b31bf660b1399bbd'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:33.886541 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1339/2693] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:33 2022] DELETE /identity/v3/users/da297351b65f4fa1b31bf660b1399bbd => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:34.228056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76110cd3-6e32-491b-804a-4cb12383be4c service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:34.242634 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76110cd3-6e32-491b-804a-4cb12383be4c service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:34.277350 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500fa5c-b394-4e93-8a54-a988bafcfa0d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:34.279128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5500fa5c-b394-4e93-8a54-a988bafcfa0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:34.334379 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad94771-b21c-4d93-a8f7-ba77d0d7efb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:34.335148 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad94771-b21c-4d93-a8f7-ba77d0d7efb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:34.335492 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad94771-b21c-4d93-a8f7-ba77d0d7efb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:34.335867 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad94771-b21c-4d93-a8f7-ba77d0d7efb1 admin admin] PATH_INFO: `/v3/projects/78b3cab8e7094e8d8c07643a948c6b46` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:34.337301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad94771-b21c-4d93-a8f7-ba77d0d7efb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78b3cab8e7094e8d8c07643a948c6b46)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:34.337661 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:34.339616 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad94771-b21c-4d93-a8f7-ba77d0d7efb1 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:34.367472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500fa5c-b394-4e93-8a54-a988bafcfa0d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:34.375641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76999f2-3c5e-48da-849c-435c53224d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:34.376735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76999f2-3c5e-48da-849c-435c53224d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:34.377836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76999f2-3c5e-48da-849c-435c53224d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:34.377836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76999f2-3c5e-48da-849c-435c53224d9d admin admin] PATH_INFO: `/v3/projects/edce948f32c641f9bd25fc38ec1742be` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:34.379001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76999f2-3c5e-48da-849c-435c53224d9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=edce948f32c641f9bd25fc38ec1742be)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:34.381082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76999f2-3c5e-48da-849c-435c53224d9d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:34.406370 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8ad94771-b21c-4d93-a8f7-ba77d0d7efb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78b3cab8e7094e8d8c07643a948c6b46', 'request_id': 'req-8ad94771-b21c-4d93-a8f7-ba77d0d7efb1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:34.409176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:34.416865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8ad94771-b21c-4d93-a8f7-ba77d0d7efb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78b3cab8e7094e8d8c07643a948c6b46', 'request_id': 'req-8ad94771-b21c-4d93-a8f7-ba77d0d7efb1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:34.430940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8ad94771-b21c-4d93-a8f7-ba77d0d7efb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78b3cab8e7094e8d8c07643a948c6b46', 'request_id': 'req-8ad94771-b21c-4d93-a8f7-ba77d0d7efb1'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:34.440499 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b76999f2-3c5e-48da-849c-435c53224d9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edce948f32c641f9bd25fc38ec1742be', 'request_id': 'req-b76999f2-3c5e-48da-849c-435c53224d9d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:34.445408 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8ad94771-b21c-4d93-a8f7-ba77d0d7efb1 admin admin] The token cache is being invalidate because project 78b3cab8e7094e8d8c07643a948c6b46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:34.445705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8ad94771-b21c-4d93-a8f7-ba77d0d7efb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:34.446462 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-8ad94771-b21c-4d93-a8f7-ba77d0d7efb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:34.447993 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b76999f2-3c5e-48da-849c-435c53224d9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edce948f32c641f9bd25fc38ec1742be', 'request_id': 'req-b76999f2-3c5e-48da-849c-435c53224d9d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:34.448855 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1355/2694] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:34 2022] DELETE /identity/v3/projects/78b3cab8e7094e8d8c07643a948c6b46 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:34.454076 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44582632-893a-4ad9-9565-1a90860b292a None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:34.454770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44582632-893a-4ad9-9565-1a90860b292a None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:34.455171 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44582632-893a-4ad9-9565-1a90860b292a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:34.459564 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b76999f2-3c5e-48da-849c-435c53224d9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edce948f32c641f9bd25fc38ec1742be', 'request_id': 'req-b76999f2-3c5e-48da-849c-435c53224d9d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:34.475165 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b76999f2-3c5e-48da-849c-435c53224d9d admin admin] The token cache is being invalidate because project edce948f32c641f9bd25fc38ec1742be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:34.475378 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b76999f2-3c5e-48da-849c-435c53224d9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:34.476029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-b76999f2-3c5e-48da-849c-435c53224d9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:34.477600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1340/2695] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:34 2022] DELETE /identity/v3/projects/edce948f32c641f9bd25fc38ec1742be => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:34.481843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-44582632-893a-4ad9-9565-1a90860b292a None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:34.482023 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76999f2-3c5e-48da-849c-435c53224d9d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:34.483327 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-44582632-893a-4ad9-9565-1a90860b292a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:34.483486 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b76999f2-3c5e-48da-849c-435c53224d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:34.551933 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1356/2696] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:34.555406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76999f2-3c5e-48da-849c-435c53224d9d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:34.560100 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad1dd1-b649-4087-a055-44256998f077 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:34.560337 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d63aa7-f674-45db-a48b-6c5df3b47fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:34.560514 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad1dd1-b649-4087-a055-44256998f077 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:34.560699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad1dd1-b649-4087-a055-44256998f077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:34.560972 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d63aa7-f674-45db-a48b-6c5df3b47fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:34.561277 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d63aa7-f674-45db-a48b-6c5df3b47fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:34.561548 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d63aa7-f674-45db-a48b-6c5df3b47fa8 admin admin] PATH_INFO: `/v3/users/a7f8885301d64b82add03032cdb0930c` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:34.562720 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d63aa7-f674-45db-a48b-6c5df3b47fa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7f8885301d64b82add03032cdb0930c)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:34.564662 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d63aa7-f674-45db-a48b-6c5df3b47fa8 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:34.580536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-ebad1dd1-b649-4087-a055-44256998f077 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:34.581765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ebad1dd1-b649-4087-a055-44256998f077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:34.589921 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1357/2697] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:34.594161 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad1dd1-b649-4087-a055-44256998f077 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:34.605826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad1dd1-b649-4087-a055-44256998f077 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:34.610679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b3b37-2be9-4346-bfdb-b57762509d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:34.611035 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b3b37-2be9-4346-bfdb-b57762509d19 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:34.611312 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b3b37-2be9-4346-bfdb-b57762509d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:34.611734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b3b37-2be9-4346-bfdb-b57762509d19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:34.612383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b3b37-2be9-4346-bfdb-b57762509d19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:56:34.612716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b3b37-2be9-4346-bfdb-b57762509d19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:34.614444 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b3b37-2be9-4346-bfdb-b57762509d19 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:34.618835 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-73d63aa7-f674-45db-a48b-6c5df3b47fa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7f8885301d64b82add03032cdb0930c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:34.626796 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1358/2698] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:56:34.628146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-73d63aa7-f674-45db-a48b-6c5df3b47fa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7f8885301d64b82add03032cdb0930c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:34.628879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-73d63aa7-f674-45db-a48b-6c5df3b47fa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7f8885301d64b82add03032cdb0930c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:34.629640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-73d63aa7-f674-45db-a48b-6c5df3b47fa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7f8885301d64b82add03032cdb0930c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:34.631143 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b3b37-2be9-4346-bfdb-b57762509d19 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:34.633003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5e7b3b37-2be9-4346-bfdb-b57762509d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:34.642532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:34.644236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-73d63aa7-f674-45db-a48b-6c5df3b47fa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7f8885301d64b82add03032cdb0930c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:34.646311 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b3b37-2be9-4346-bfdb-b57762509d19 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:34.651426 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171631ce-e980-4936-b83d-2e2ce06c0879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:34.652074 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171631ce-e980-4936-b83d-2e2ce06c0879 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:34.652355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171631ce-e980-4936-b83d-2e2ce06c0879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:34.652630 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171631ce-e980-4936-b83d-2e2ce06c0879 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:34.653363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171631ce-e980-4936-b83d-2e2ce06c0879 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:34.655284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171631ce-e980-4936-b83d-2e2ce06c0879 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:34.658637 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1341/2699] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:34 2022] DELETE /identity/v3/users/a7f8885301d64b82add03032cdb0930c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:34.684061 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1359/2700] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:34 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:34.688120 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d63aa7-f674-45db-a48b-6c5df3b47fa8 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:34.752179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d63aa7-f674-45db-a48b-6c5df3b47fa8 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:34.818893 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce1ec7-51ca-4052-b946-bf0d7fbfb045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:34.819516 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce1ec7-51ca-4052-b946-bf0d7fbfb045 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:34.819782 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce1ec7-51ca-4052-b946-bf0d7fbfb045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:34.820053 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce1ec7-51ca-4052-b946-bf0d7fbfb045 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:34.820725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ce1ec7-51ca-4052-b946-bf0d7fbfb045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:34.822759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ce1ec7-51ca-4052-b946-bf0d7fbfb045 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:34.847829 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1342/2701] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:34 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:34.852306 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171631ce-e980-4936-b83d-2e2ce06c0879 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:34.864757 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171631ce-e980-4936-b83d-2e2ce06c0879 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:34.871202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336757c-6500-4602-b76f-e9d57af5760c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:34.871824 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336757c-6500-4602-b76f-e9d57af5760c admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:34.872121 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336757c-6500-4602-b76f-e9d57af5760c admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:34.872391 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336757c-6500-4602-b76f-e9d57af5760c admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:34.873039 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336757c-6500-4602-b76f-e9d57af5760c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:34.874674 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336757c-6500-4602-b76f-e9d57af5760c admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:34.885882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1360/2702] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:34.889749 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce1ec7-51ca-4052-b946-bf0d7fbfb045 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:34.900231 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce1ec7-51ca-4052-b946-bf0d7fbfb045 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:34.906002 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c332662-3761-4728-bd86-bc8e71d54874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:34.907020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c332662-3761-4728-bd86-bc8e71d54874 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:34.907583 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c332662-3761-4728-bd86-bc8e71d54874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:34.908100 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c332662-3761-4728-bd86-bc8e71d54874 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:34.909307 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c332662-3761-4728-bd86-bc8e71d54874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:34.912489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c332662-3761-4728-bd86-bc8e71d54874 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:34.927899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1343/2703] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:34.929166 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336757c-6500-4602-b76f-e9d57af5760c admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:34.940024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336757c-6500-4602-b76f-e9d57af5760c admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:34.944740 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2e4d5-e26a-4195-a5d8-5e5c24e3ce67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:34.945409 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2e4d5-e26a-4195-a5d8-5e5c24e3ce67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:34.945688 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2e4d5-e26a-4195-a5d8-5e5c24e3ce67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:34.946117 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2e4d5-e26a-4195-a5d8-5e5c24e3ce67 admin admin] PATH_INFO: `/v3/projects/d4bad89628aa410ab4c5c10a55ec0e5c/users/2385fae8b88044719313a590bd72817d/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:34.961528 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2e4d5-e26a-4195-a5d8-5e5c24e3ce67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4bad89628aa410ab4c5c10a55ec0e5c, user_id=2385fae8b88044719313a590bd72817d, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:34.964207 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2e4d5-e26a-4195-a5d8-5e5c24e3ce67 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:34.979028 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1361/2704] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:34 2022] PUT /identity/v3/projects/d4bad89628aa410ab4c5c10a55ec0e5c/users/2385fae8b88044719313a590bd72817d/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:34.991505 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b69985-885d-430a-8d84-8aac63e80c61 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:34.991797 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b69985-885d-430a-8d84-8aac63e80c61 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:34.992074 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b69985-885d-430a-8d84-8aac63e80c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:34.996065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2e4d5-e26a-4195-a5d8-5e5c24e3ce67 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:34.998073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-0fa2e4d5-e26a-4195-a5d8-5e5c24e3ce67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:35.026823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-21b69985-885d-430a-8d84-8aac63e80c61 None None] MFA Rules not processed for user `2385fae8b88044719313a590bd72817d`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:35.028105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-21b69985-885d-430a-8d84-8aac63e80c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:35.069908 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2e4d5-e26a-4195-a5d8-5e5c24e3ce67 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:35.074582 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca492e5-c004-4ec6-bfe9-6263f300cc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:35.075235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca492e5-c004-4ec6-bfe9-6263f300cc4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.075479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca492e5-c004-4ec6-bfe9-6263f300cc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.075748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca492e5-c004-4ec6-bfe9-6263f300cc4c admin admin] PATH_INFO: `/v3/projects/560ff79b6fcd462bb236a45953bedbc1` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.076875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca492e5-c004-4ec6-bfe9-6263f300cc4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=560ff79b6fcd462bb236a45953bedbc1)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:35.078878 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca492e5-c004-4ec6-bfe9-6263f300cc4c admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:35.086753 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:35.117550 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1344/2705] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:35.122295 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b69985-885d-430a-8d84-8aac63e80c61 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:35.128726 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:35.135034 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0ca492e5-c004-4ec6-bfe9-6263f300cc4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '560ff79b6fcd462bb236a45953bedbc1', 'request_id': 'req-0ca492e5-c004-4ec6-bfe9-6263f300cc4c'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:35.138836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b69985-885d-430a-8d84-8aac63e80c61 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:35.142256 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0ca492e5-c004-4ec6-bfe9-6263f300cc4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '560ff79b6fcd462bb236a45953bedbc1', 'request_id': 'req-0ca492e5-c004-4ec6-bfe9-6263f300cc4c'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:35.151573 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0ca492e5-c004-4ec6-bfe9-6263f300cc4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '560ff79b6fcd462bb236a45953bedbc1', 'request_id': 'req-0ca492e5-c004-4ec6-bfe9-6263f300cc4c'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:35.165974 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0ca492e5-c004-4ec6-bfe9-6263f300cc4c admin admin] The token cache is being invalidate because project 560ff79b6fcd462bb236a45953bedbc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:35.166638 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0ca492e5-c004-4ec6-bfe9-6263f300cc4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:35.167488 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0ca492e5-c004-4ec6-bfe9-6263f300cc4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:35.169119 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1362/2706] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:34 2022] DELETE /identity/v3/projects/560ff79b6fcd462bb236a45953bedbc1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:35.174245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ff86b-deb6-4376-9f7a-8b7594e125ea None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.174406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ff86b-deb6-4376-9f7a-8b7594e125ea None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.174583 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ff86b-deb6-4376-9f7a-8b7594e125ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.194632 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-7c3ff86b-deb6-4376-9f7a-8b7594e125ea None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:35.195863 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7c3ff86b-deb6-4376-9f7a-8b7594e125ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:35.206205 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098b142-c953-4dd7-9b16-348b872d60ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:35.206825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098b142-c953-4dd7-9b16-348b872d60ca admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.207100 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098b142-c953-4dd7-9b16-348b872d60ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.207365 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098b142-c953-4dd7-9b16-348b872d60ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.208004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098b142-c953-4dd7-9b16-348b872d60ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:35.210104 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098b142-c953-4dd7-9b16-348b872d60ca admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:35.243663 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1345/2707] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:35 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:35.248028 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098b142-c953-4dd7-9b16-348b872d60ca admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:35.249809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7098b142-c953-4dd7-9b16-348b872d60ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:35.267577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1363/2708] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:35.275245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfc987-8971-4bea-a0c4-723de92c6dd2 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.275532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfc987-8971-4bea-a0c4-723de92c6dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.275807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfc987-8971-4bea-a0c4-723de92c6dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.295674 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-dfbfc987-8971-4bea-a0c4-723de92c6dd2 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:35.296861 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-dfbfc987-8971-4bea-a0c4-723de92c6dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:35.316534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098b142-c953-4dd7-9b16-348b872d60ca admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:35.321265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373f3f0-ff4e-450b-8aa6-17ae61eabf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:35.321888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373f3f0-ff4e-450b-8aa6-17ae61eabf3e admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.322162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373f3f0-ff4e-450b-8aa6-17ae61eabf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.322438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373f3f0-ff4e-450b-8aa6-17ae61eabf3e admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.323075 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373f3f0-ff4e-450b-8aa6-17ae61eabf3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:35.324991 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373f3f0-ff4e-450b-8aa6-17ae61eabf3e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:35.347931 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1346/2709] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:35 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:35.352029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373f3f0-ff4e-450b-8aa6-17ae61eabf3e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:35.360605 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1364/2710] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:35.364302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373f3f0-ff4e-450b-8aa6-17ae61eabf3e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:35.365076 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbfc987-8971-4bea-a0c4-723de92c6dd2 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:35.367047 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0714c2-3cad-4c95-95e6-7568fa169886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:35.367726 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0714c2-3cad-4c95-95e6-7568fa169886 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.368073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0714c2-3cad-4c95-95e6-7568fa169886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.368411 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0714c2-3cad-4c95-95e6-7568fa169886 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.369109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0714c2-3cad-4c95-95e6-7568fa169886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:35.370822 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0714c2-3cad-4c95-95e6-7568fa169886 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:35.374500 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbfc987-8971-4bea-a0c4-723de92c6dd2 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:35.379765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc3ddb-d3a9-4543-bd14-392519040f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:35.380366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc3ddb-d3a9-4543-bd14-392519040f72 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.380642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc3ddb-d3a9-4543-bd14-392519040f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.380912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc3ddb-d3a9-4543-bd14-392519040f72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.381611 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc3ddb-d3a9-4543-bd14-392519040f72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81663) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:56:35.381915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc3ddb-d3a9-4543-bd14-392519040f72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:35.382073 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1347/2711] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:35.383503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc3ddb-d3a9-4543-bd14-392519040f72 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:35.386454 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0714c2-3cad-4c95-95e6-7568fa169886 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:35.395902 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1365/2712] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:56:35.399433 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0714c2-3cad-4c95-95e6-7568fa169886 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:35.400427 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc3ddb-d3a9-4543-bd14-392519040f72 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:35.402040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2cdcf-fe40-4838-a4ee-f081daf68b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:35.402695 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2cdcf-fe40-4838-a4ee-f081daf68b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.402982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2cdcf-fe40-4838-a4ee-f081daf68b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.403240 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2cdcf-fe40-4838-a4ee-f081daf68b33 admin admin] PATH_INFO: `/v3/projects/ed7bd89142914178b14621ea50b39d91/users/60ddd59aec66429697baeb4221a35c19/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.411524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc3ddb-d3a9-4543-bd14-392519040f72 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:35.416419 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b762e81-210f-4985-b67e-d76a31875432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:35.417018 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b762e81-210f-4985-b67e-d76a31875432 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.417347 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b762e81-210f-4985-b67e-d76a31875432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.417624 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b762e81-210f-4985-b67e-d76a31875432 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.418256 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b762e81-210f-4985-b67e-d76a31875432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:35.419906 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2cdcf-fe40-4838-a4ee-f081daf68b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7bd89142914178b14621ea50b39d91, user_id=60ddd59aec66429697baeb4221a35c19, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:35.420179 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b762e81-210f-4985-b67e-d76a31875432 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:35.422571 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2cdcf-fe40-4838-a4ee-f081daf68b33 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:35.436965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1348/2713] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:35 2022] PUT /identity/v3/projects/ed7bd89142914178b14621ea50b39d91/users/60ddd59aec66429697baeb4221a35c19/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:35.441212 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2cdcf-fe40-4838-a4ee-f081daf68b33 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:35.447241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:35.458835 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1366/2714] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:35 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:35.459793 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b762e81-210f-4985-b67e-d76a31875432 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:35.486231 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:35.504263 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2cdcf-fe40-4838-a4ee-f081daf68b33 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:35.526025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b762e81-210f-4985-b67e-d76a31875432 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:35.531570 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2556c23-ed48-4f47-afad-fd66ffbb190f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:35.532186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2556c23-ed48-4f47-afad-fd66ffbb190f admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.532461 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2556c23-ed48-4f47-afad-fd66ffbb190f admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.532727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2556c23-ed48-4f47-afad-fd66ffbb190f admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.533391 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2556c23-ed48-4f47-afad-fd66ffbb190f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:35.535479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2556c23-ed48-4f47-afad-fd66ffbb190f admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:35.563113 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1367/2715] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:35 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:35.567473 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2556c23-ed48-4f47-afad-fd66ffbb190f admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:35.568475 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4a493-5107-4c33-bf15-d1c7c90078d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:35.569126 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4a493-5107-4c33-bf15-d1c7c90078d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.569532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4a493-5107-4c33-bf15-d1c7c90078d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.569825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4a493-5107-4c33-bf15-d1c7c90078d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.570469 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4a493-5107-4c33-bf15-d1c7c90078d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:35.572070 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4a493-5107-4c33-bf15-d1c7c90078d2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:35.578433 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2556c23-ed48-4f47-afad-fd66ffbb190f admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:35.583105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1349/2716] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:35.587057 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f277d-62f9-48a6-9548-f6212c64b01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:35.587658 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4a493-5107-4c33-bf15-d1c7c90078d2 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:35.587772 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f277d-62f9-48a6-9548-f6212c64b01b admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.587940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f277d-62f9-48a6-9548-f6212c64b01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.588209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f277d-62f9-48a6-9548-f6212c64b01b admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.588964 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f277d-62f9-48a6-9548-f6212c64b01b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:35.590711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f277d-62f9-48a6-9548-f6212c64b01b admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:35.600039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4a493-5107-4c33-bf15-d1c7c90078d2 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:35.601875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1368/2717] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:35.606109 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f277d-62f9-48a6-9548-f6212c64b01b admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:35.608639 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb5952-251c-44b3-babb-a4e6969b1267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:35.609543 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb5952-251c-44b3-babb-a4e6969b1267 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.609925 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb5952-251c-44b3-babb-a4e6969b1267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.610265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb5952-251c-44b3-babb-a4e6969b1267 admin admin] PATH_INFO: `/v3/projects/ed7bd89142914178b14621ea50b39d91/users/60ddd59aec66429697baeb4221a35c19/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.613359 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb5952-251c-44b3-babb-a4e6969b1267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7bd89142914178b14621ea50b39d91, user_id=60ddd59aec66429697baeb4221a35c19, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:35.616070 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb5952-251c-44b3-babb-a4e6969b1267 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:35.616493 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f277d-62f9-48a6-9548-f6212c64b01b admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:35.622127 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd63f9-0a89-4c89-a0c0-8b6c7b172201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:35.623095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd63f9-0a89-4c89-a0c0-8b6c7b172201 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.623694 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd63f9-0a89-4c89-a0c0-8b6c7b172201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.624031 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd63f9-0a89-4c89-a0c0-8b6c7b172201 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.624761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd63f9-0a89-4c89-a0c0-8b6c7b172201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:35.626601 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd63f9-0a89-4c89-a0c0-8b6c7b172201 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:35.628004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1350/2718] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:35 2022] PUT /identity/v3/projects/ed7bd89142914178b14621ea50b39d91/users/60ddd59aec66429697baeb4221a35c19/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:35.638299 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73ab24-c3d1-445e-b465-b4835b9fabcb None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.638992 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73ab24-c3d1-445e-b465-b4835b9fabcb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.639082 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1369/2719] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:35.641333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73ab24-c3d1-445e-b465-b4835b9fabcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.644987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd63f9-0a89-4c89-a0c0-8b6c7b172201 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:35.655434 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd63f9-0a89-4c89-a0c0-8b6c7b172201 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:35.671714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8d73ab24-c3d1-445e-b465-b4835b9fabcb None None] MFA Rules not processed for user `60ddd59aec66429697baeb4221a35c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:35.673008 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8d73ab24-c3d1-445e-b465-b4835b9fabcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:35.724558 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7daf3-5ade-49a3-8be5-378c7c4ce39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:35.725255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7daf3-5ade-49a3-8be5-378c7c4ce39a admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.725530 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7daf3-5ade-49a3-8be5-378c7c4ce39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.725791 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7daf3-5ade-49a3-8be5-378c7c4ce39a admin admin] PATH_INFO: `/v3/projects/7abad03e64cb47fba3d37d5f84b3c899/users/8d82b4df4e7c49b38aac207507be6f1e/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.740227 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7daf3-5ade-49a3-8be5-378c7c4ce39a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7abad03e64cb47fba3d37d5f84b3c899, user_id=8d82b4df4e7c49b38aac207507be6f1e, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:35.743066 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7daf3-5ade-49a3-8be5-378c7c4ce39a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:35.753727 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1351/2720] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:35.760082 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1370/2721] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:35 2022] PUT /identity/v3/projects/7abad03e64cb47fba3d37d5f84b3c899/users/8d82b4df4e7c49b38aac207507be6f1e/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:35.765020 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73ab24-c3d1-445e-b465-b4835b9fabcb None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:35.766549 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8d73ab24-c3d1-445e-b465-b4835b9fabcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:35.771322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4b871-0bee-43e9-a198-320525071427 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.771618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4b871-0bee-43e9-a198-320525071427 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.771910 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4b871-0bee-43e9-a198-320525071427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.799643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d6f4b871-0bee-43e9-a198-320525071427 None None] MFA Rules not processed for user `8d82b4df4e7c49b38aac207507be6f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:35.800863 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d6f4b871-0bee-43e9-a198-320525071427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:35.824718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73ab24-c3d1-445e-b465-b4835b9fabcb None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:35.834637 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:35.844395 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:35.877622 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1371/2722] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:35.878642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4b871-0bee-43e9-a198-320525071427 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:35.882666 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25945e-af76-4369-a073-85da0e3861cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:35.883288 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25945e-af76-4369-a073-85da0e3861cd service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.883569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25945e-af76-4369-a073-85da0e3861cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.883841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25945e-af76-4369-a073-85da0e3861cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.941146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4b871-0bee-43e9-a198-320525071427 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:35.945881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319f544-07f9-4a12-8bbe-655f67042157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:35.946341 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25945e-af76-4369-a073-85da0e3861cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:35.946473 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319f544-07f9-4a12-8bbe-655f67042157 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:35.946765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319f544-07f9-4a12-8bbe-655f67042157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:35.947017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319f544-07f9-4a12-8bbe-655f67042157 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:35.947631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319f544-07f9-4a12-8bbe-655f67042157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:35.947755 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25945e-af76-4369-a073-85da0e3861cd service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:35.949605 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319f544-07f9-4a12-8bbe-655f67042157 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:35.958823 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1352/2723] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:35 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:35.980541 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1372/2724] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:35 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:35.985652 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25945e-af76-4369-a073-85da0e3861cd service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:35.996147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25945e-af76-4369-a073-85da0e3861cd service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:36.055420 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72116b-bfc6-425e-9c16-6f15d4cf0344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:36.056145 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72116b-bfc6-425e-9c16-6f15d4cf0344 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:36.056363 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72116b-bfc6-425e-9c16-6f15d4cf0344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:36.056636 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72116b-bfc6-425e-9c16-6f15d4cf0344 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:36.057327 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b72116b-bfc6-425e-9c16-6f15d4cf0344 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:36.059338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b72116b-bfc6-425e-9c16-6f15d4cf0344 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:36.088305 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1353/2725] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:35 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:36.089791 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319f544-07f9-4a12-8bbe-655f67042157 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:36.106189 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319f544-07f9-4a12-8bbe-655f67042157 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:36.110294 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1e7b0-bf46-4e09-9c06-74336ad9db18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:36.110928 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1e7b0-bf46-4e09-9c06-74336ad9db18 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:36.111203 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1e7b0-bf46-4e09-9c06-74336ad9db18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:36.111462 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1e7b0-bf46-4e09-9c06-74336ad9db18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:36.112116 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1e7b0-bf46-4e09-9c06-74336ad9db18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:36.113735 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1e7b0-bf46-4e09-9c06-74336ad9db18 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:36.125587 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1373/2726] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:36.130060 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72116b-bfc6-425e-9c16-6f15d4cf0344 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:36.144540 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72116b-bfc6-425e-9c16-6f15d4cf0344 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:36.150906 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca022d32-d3ed-4513-b88e-951e4e369f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:36.151542 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca022d32-d3ed-4513-b88e-951e4e369f6c admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:36.151927 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca022d32-d3ed-4513-b88e-951e4e369f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:36.152195 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca022d32-d3ed-4513-b88e-951e4e369f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:36.152840 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca022d32-d3ed-4513-b88e-951e4e369f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:36.154467 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca022d32-d3ed-4513-b88e-951e4e369f6c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:36.170092 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1354/2727] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:36.175269 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1e7b0-bf46-4e09-9c06-74336ad9db18 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:36.186477 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1e7b0-bf46-4e09-9c06-74336ad9db18 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:36.191756 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bdcf0-e585-412a-b68a-a2b45430dc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:36.192435 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bdcf0-e585-412a-b68a-a2b45430dc4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:36.192711 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bdcf0-e585-412a-b68a-a2b45430dc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:36.192984 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bdcf0-e585-412a-b68a-a2b45430dc4c admin admin] PATH_INFO: `/v3/projects/d7e4def861fe47e2ba5b954524777182/users/6042f3363bb64626ab3499582f47eb18/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:36.209898 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bdcf0-e585-412a-b68a-a2b45430dc4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7e4def861fe47e2ba5b954524777182, user_id=6042f3363bb64626ab3499582f47eb18, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:36.212602 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bdcf0-e585-412a-b68a-a2b45430dc4c admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:36.230526 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1374/2728] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:36 2022] PUT /identity/v3/projects/d7e4def861fe47e2ba5b954524777182/users/6042f3363bb64626ab3499582f47eb18/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:36.241541 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb9844-ffb6-41f0-9963-53306c308e41 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:36.241821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb9844-ffb6-41f0-9963-53306c308e41 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:36.242087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb9844-ffb6-41f0-9963-53306c308e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:36.272984 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-9eeb9844-ffb6-41f0-9963-53306c308e41 None None] MFA Rules not processed for user `6042f3363bb64626ab3499582f47eb18`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:36.274260 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9eeb9844-ffb6-41f0-9963-53306c308e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:36.308513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:36.346704 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1355/2729] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:36.352381 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bdcf0-e585-412a-b68a-a2b45430dc4c admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:36.423082 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bdcf0-e585-412a-b68a-a2b45430dc4c admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:36.428628 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515c691-a62b-4ab2-ba3c-c5ccf37486bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:36.429304 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515c691-a62b-4ab2-ba3c-c5ccf37486bf admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:36.429584 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515c691-a62b-4ab2-ba3c-c5ccf37486bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:36.430139 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515c691-a62b-4ab2-ba3c-c5ccf37486bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:36.430499 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515c691-a62b-4ab2-ba3c-c5ccf37486bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:36.432860 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515c691-a62b-4ab2-ba3c-c5ccf37486bf admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:36.472154 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1375/2730] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:36 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:36.478069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb9844-ffb6-41f0-9963-53306c308e41 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:36.495726 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb9844-ffb6-41f0-9963-53306c308e41 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:36.559805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aaff5a-c5c1-4619-94c6-fab059149f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:36.560455 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaff5a-c5c1-4619-94c6-fab059149f37 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:36.560686 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaff5a-c5c1-4619-94c6-fab059149f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:36.560945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaff5a-c5c1-4619-94c6-fab059149f37 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:36.561628 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aaff5a-c5c1-4619-94c6-fab059149f37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:36.563635 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aaff5a-c5c1-4619-94c6-fab059149f37 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:36.566485 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515c691-a62b-4ab2-ba3c-c5ccf37486bf admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:36.579054 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515c691-a62b-4ab2-ba3c-c5ccf37486bf admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:36.586571 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875fdf0-43a5-4d6f-99a1-39730120605c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:36.587776 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875fdf0-43a5-4d6f-99a1-39730120605c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:36.588114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875fdf0-43a5-4d6f-99a1-39730120605c admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:36.588843 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875fdf0-43a5-4d6f-99a1-39730120605c admin admin] PATH_INFO: `/v3/users/2385fae8b88044719313a590bd72817d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:36.592680 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1356/2731] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:36 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:36.592871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6875fdf0-43a5-4d6f-99a1-39730120605c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2385fae8b88044719313a590bd72817d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:36.592871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6875fdf0-43a5-4d6f-99a1-39730120605c admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:36.594060 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aaff5a-c5c1-4619-94c6-fab059149f37 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:36.607415 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:36.612186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aaff5a-c5c1-4619-94c6-fab059149f37 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:36.618954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca8620-89f7-47a9-814e-42a3af85fbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:36.619602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca8620-89f7-47a9-814e-42a3af85fbc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:36.619877 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca8620-89f7-47a9-814e-42a3af85fbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:36.620147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca8620-89f7-47a9-814e-42a3af85fbc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:36.621576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca8620-89f7-47a9-814e-42a3af85fbc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:36.624439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca8620-89f7-47a9-814e-42a3af85fbc1 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:36.637125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1357/2732] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:36.641788 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca8620-89f7-47a9-814e-42a3af85fbc1 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:36.653797 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca8620-89f7-47a9-814e-42a3af85fbc1 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:36.658003 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6875fdf0-43a5-4d6f-99a1-39730120605c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2385fae8b88044719313a590bd72817d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:36.658744 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47745d26-a8e6-4c00-9a4a-ce1ad025ce32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:36.659566 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47745d26-a8e6-4c00-9a4a-ce1ad025ce32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:36.659774 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47745d26-a8e6-4c00-9a4a-ce1ad025ce32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:36.660220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47745d26-a8e6-4c00-9a4a-ce1ad025ce32 admin admin] PATH_INFO: `/v3/projects/6b9cd3e7f77c48db874a151d9ab07a21/users/262b5b4ecd8f4a94a35b6714acc42d5a/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:36.670968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6875fdf0-43a5-4d6f-99a1-39730120605c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2385fae8b88044719313a590bd72817d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:36.671674 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6875fdf0-43a5-4d6f-99a1-39730120605c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2385fae8b88044719313a590bd72817d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:36.672356 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6875fdf0-43a5-4d6f-99a1-39730120605c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2385fae8b88044719313a590bd72817d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:36.678421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47745d26-a8e6-4c00-9a4a-ce1ad025ce32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b9cd3e7f77c48db874a151d9ab07a21, user_id=262b5b4ecd8f4a94a35b6714acc42d5a, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:36.680310 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6875fdf0-43a5-4d6f-99a1-39730120605c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2385fae8b88044719313a590bd72817d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:36.680446 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47745d26-a8e6-4c00-9a4a-ce1ad025ce32 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:36.696658 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1358/2733] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:36 2022] PUT /identity/v3/projects/6b9cd3e7f77c48db874a151d9ab07a21/users/262b5b4ecd8f4a94a35b6714acc42d5a/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:36.699815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1376/2734] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:36 2022] DELETE /identity/v3/users/2385fae8b88044719313a590bd72817d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:36.703610 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47745d26-a8e6-4c00-9a4a-ce1ad025ce32 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:36.704185 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a41988-383a-4424-8074-8dfac20e3f9d None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:36.705005 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a41988-383a-4424-8074-8dfac20e3f9d None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:36.705431 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a41988-383a-4424-8074-8dfac20e3f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:36.705789 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-47745d26-a8e6-4c00-9a4a-ce1ad025ce32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:36.731479 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-79a41988-383a-4424-8074-8dfac20e3f9d None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:36.733837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-79a41988-383a-4424-8074-8dfac20e3f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:36.793085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47745d26-a8e6-4c00-9a4a-ce1ad025ce32 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:36.814341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1377/2735] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:36.825559 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a41988-383a-4424-8074-8dfac20e3f9d None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:36.827170 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-79a41988-383a-4424-8074-8dfac20e3f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:36.828882 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:36.870779 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78173ed-16cf-4f16-af11-adcd3a1e2ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:36.871646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78173ed-16cf-4f16-af11-adcd3a1e2ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:36.871922 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78173ed-16cf-4f16-af11-adcd3a1e2ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:36.872191 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78173ed-16cf-4f16-af11-adcd3a1e2ed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:36.872831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78173ed-16cf-4f16-af11-adcd3a1e2ed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:36.874663 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78173ed-16cf-4f16-af11-adcd3a1e2ed9 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:36.891247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1359/2736] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:36.896300 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78173ed-16cf-4f16-af11-adcd3a1e2ed9 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:36.907326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78173ed-16cf-4f16-af11-adcd3a1e2ed9 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:36.907699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a41988-383a-4424-8074-8dfac20e3f9d None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:36.912655 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a95ca-35f0-4533-85ee-11e812174804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:36.913087 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfb028-9e84-4f41-b908-0a688dfd99d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:36.913503 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a95ca-35f0-4533-85ee-11e812174804 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:36.914138 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a95ca-35f0-4533-85ee-11e812174804 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:36.914262 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfb028-9e84-4f41-b908-0a688dfd99d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:36.914262 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfb028-9e84-4f41-b908-0a688dfd99d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:36.914451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a95ca-35f0-4533-85ee-11e812174804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:36.914646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfb028-9e84-4f41-b908-0a688dfd99d2 admin admin] PATH_INFO: `/v3/projects/6b9cd3e7f77c48db874a151d9ab07a21/users/262b5b4ecd8f4a94a35b6714acc42d5a/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:36.916619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dfb028-9e84-4f41-b908-0a688dfd99d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b9cd3e7f77c48db874a151d9ab07a21, user_id=262b5b4ecd8f4a94a35b6714acc42d5a, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:36.919390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dfb028-9e84-4f41-b908-0a688dfd99d2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:36.925670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a95ca-35f0-4533-85ee-11e812174804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:36.927219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a95ca-35f0-4533-85ee-11e812174804 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:36.932085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1360/2737] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:36 2022] PUT /identity/v3/projects/6b9cd3e7f77c48db874a151d9ab07a21/users/262b5b4ecd8f4a94a35b6714acc42d5a/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:36.938929 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1378/2738] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:56:36.944034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30a4b6-029a-4e43-bcd7-db68a7a16553 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:36.944034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30a4b6-029a-4e43-bcd7-db68a7a16553 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:36.944034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30a4b6-029a-4e43-bcd7-db68a7a16553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:36.985709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-fe30a4b6-029a-4e43-bcd7-db68a7a16553 None None] MFA Rules not processed for user `262b5b4ecd8f4a94a35b6714acc42d5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:36.987660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-fe30a4b6-029a-4e43-bcd7-db68a7a16553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:37.062701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1361/2739] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:37.073560 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a95ca-35f0-4533-85ee-11e812174804 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:37.075008 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-592a95ca-35f0-4533-85ee-11e812174804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:37.128631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:37.156692 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a95ca-35f0-4533-85ee-11e812174804 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:37.208833 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad1976-4cc2-4fb2-b4cf-2f86ee74d148 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:37.209507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad1976-4cc2-4fb2-b4cf-2f86ee74d148 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:37.209785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad1976-4cc2-4fb2-b4cf-2f86ee74d148 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:37.210049 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad1976-4cc2-4fb2-b4cf-2f86ee74d148 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:37.221026 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad1976-4cc2-4fb2-b4cf-2f86ee74d148 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:37.222680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad1976-4cc2-4fb2-b4cf-2f86ee74d148 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:37.226159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30a4b6-029a-4e43-bcd7-db68a7a16553 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:37.227742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-fe30a4b6-029a-4e43-bcd7-db68a7a16553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:37.238348 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1379/2740] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:37.312640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30a4b6-029a-4e43-bcd7-db68a7a16553 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:37.318825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:37.319404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:37.319685 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:37.319932 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714 admin admin] PATH_INFO: `/v3/projects/d4bad89628aa410ab4c5c10a55ec0e5c` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:37.321152 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4bad89628aa410ab4c5c10a55ec0e5c)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:37.323085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:37.371563 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:37.376066 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4bad89628aa410ab4c5c10a55ec0e5c', 'request_id': 'req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:37.383468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4bad89628aa410ab4c5c10a55ec0e5c', 'request_id': 'req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:37.391522 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4bad89628aa410ab4c5c10a55ec0e5c', 'request_id': 'req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:37.402596 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714 admin admin] The token cache is being invalidate because project d4bad89628aa410ab4c5c10a55ec0e5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:37.402889 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:37.403584 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:37.405199 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1362/2741] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:37 2022] DELETE /identity/v3/projects/d4bad89628aa410ab4c5c10a55ec0e5c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:37.409290 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad1976-4cc2-4fb2-b4cf-2f86ee74d148 service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:37.419835 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad1976-4cc2-4fb2-b4cf-2f86ee74d148 service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:37.479080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82349a58-1ba7-4955-83d1-a6434a13bc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:37.479715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82349a58-1ba7-4955-83d1-a6434a13bc6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:37.479992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82349a58-1ba7-4955-83d1-a6434a13bc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:37.480255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82349a58-1ba7-4955-83d1-a6434a13bc6c admin admin] PATH_INFO: `/v3/users/60ddd59aec66429697baeb4221a35c19` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:37.481394 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82349a58-1ba7-4955-83d1-a6434a13bc6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=60ddd59aec66429697baeb4221a35c19)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:37.483273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82349a58-1ba7-4955-83d1-a6434a13bc6c admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:37.532604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-82349a58-1ba7-4955-83d1-a6434a13bc6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60ddd59aec66429697baeb4221a35c19'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:37.540890 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-82349a58-1ba7-4955-83d1-a6434a13bc6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60ddd59aec66429697baeb4221a35c19'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:37.541827 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-82349a58-1ba7-4955-83d1-a6434a13bc6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60ddd59aec66429697baeb4221a35c19'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:37.542524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-82349a58-1ba7-4955-83d1-a6434a13bc6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60ddd59aec66429697baeb4221a35c19'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:37.550165 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-82349a58-1ba7-4955-83d1-a6434a13bc6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60ddd59aec66429697baeb4221a35c19'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:37.570898 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1380/2742] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:37 2022] DELETE /identity/v3/users/60ddd59aec66429697baeb4221a35c19 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:37.784246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:37.785679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:37.845004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa35e4c-f463-4cbc-864f-c25b3ddbc714 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:37.850312 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82349a58-1ba7-4955-83d1-a6434a13bc6c admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:37.852123 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-82349a58-1ba7-4955-83d1-a6434a13bc6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:37.864044 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:37.907657 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7effe90-5727-4f84-a75b-1d885d5ba107 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:37.908449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7effe90-5727-4f84-a75b-1d885d5ba107 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:37.908729 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7effe90-5727-4f84-a75b-1d885d5ba107 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:37.909004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7effe90-5727-4f84-a75b-1d885d5ba107 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:37.911046 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d7effe90-5727-4f84-a75b-1d885d5ba107 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:37.932683 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82349a58-1ba7-4955-83d1-a6434a13bc6c admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:37.937618 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d58d8-b98f-44a0-abdb-78e9f3bfc791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:37.938530 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d58d8-b98f-44a0-abdb-78e9f3bfc791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:37.938892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d58d8-b98f-44a0-abdb-78e9f3bfc791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:37.939238 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d58d8-b98f-44a0-abdb-78e9f3bfc791 admin admin] PATH_INFO: `/v3/projects/ed7bd89142914178b14621ea50b39d91` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:37.940467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d58d8-b98f-44a0-abdb-78e9f3bfc791 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed7bd89142914178b14621ea50b39d91)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:37.942490 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d58d8-b98f-44a0-abdb-78e9f3bfc791 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:37.979881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7effe90-5727-4f84-a75b-1d885d5ba107 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:37.981442 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7effe90-5727-4f84-a75b-1d885d5ba107 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:38.003631 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1363/2743] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:37 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:38.007700 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a15d58d8-b98f-44a0-abdb-78e9f3bfc791 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed7bd89142914178b14621ea50b39d91', 'request_id': 'req-a15d58d8-b98f-44a0-abdb-78e9f3bfc791'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:38.018001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a15d58d8-b98f-44a0-abdb-78e9f3bfc791 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed7bd89142914178b14621ea50b39d91', 'request_id': 'req-a15d58d8-b98f-44a0-abdb-78e9f3bfc791'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:38.031295 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a15d58d8-b98f-44a0-abdb-78e9f3bfc791 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed7bd89142914178b14621ea50b39d91', 'request_id': 'req-a15d58d8-b98f-44a0-abdb-78e9f3bfc791'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:38.049525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a15d58d8-b98f-44a0-abdb-78e9f3bfc791 admin admin] The token cache is being invalidate because project ed7bd89142914178b14621ea50b39d91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:38.050308 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a15d58d8-b98f-44a0-abdb-78e9f3bfc791 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:38.051057 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-a15d58d8-b98f-44a0-abdb-78e9f3bfc791 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:38.053341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1381/2744] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:37 2022] DELETE /identity/v3/projects/ed7bd89142914178b14621ea50b39d91 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:38.058255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ca0c9-4850-4c2f-9fdd-65be549233e9 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:38.058765 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ca0c9-4850-4c2f-9fdd-65be549233e9 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:38.059252 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ca0c9-4850-4c2f-9fdd-65be549233e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:38.088612 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-134ca0c9-4850-4c2f-9fdd-65be549233e9 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:38.091531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-134ca0c9-4850-4c2f-9fdd-65be549233e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:38.143438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:38.179117 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1364/2745] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:38.189193 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337065c9-f079-4588-8eeb-461612107750 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:38.189621 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337065c9-f079-4588-8eeb-461612107750 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:38.190024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337065c9-f079-4588-8eeb-461612107750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:38.222523 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-337065c9-f079-4588-8eeb-461612107750 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:38.223947 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-337065c9-f079-4588-8eeb-461612107750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:38.238358 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1382/2746] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:38.242744 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ca0c9-4850-4c2f-9fdd-65be549233e9 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:38.253417 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ca0c9-4850-4c2f-9fdd-65be549233e9 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:38.259534 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed7bf0-ed83-4aed-b651-75477dfd83c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:38.260184 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed7bf0-ed83-4aed-b651-75477dfd83c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:38.260455 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed7bf0-ed83-4aed-b651-75477dfd83c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:38.260849 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed7bf0-ed83-4aed-b651-75477dfd83c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:38.261447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed7bf0-ed83-4aed-b651-75477dfd83c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:56:38.261750 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed7bf0-ed83-4aed-b651-75477dfd83c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:38.263355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed7bf0-ed83-4aed-b651-75477dfd83c4 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:38.276257 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1365/2747] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:56:38.280975 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337065c9-f079-4588-8eeb-461612107750 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:38.293391 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337065c9-f079-4588-8eeb-461612107750 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:38.298605 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc3779-cf11-4143-9107-d6229ec9cdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:38.299127 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc3779-cf11-4143-9107-d6229ec9cdc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:38.299404 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc3779-cf11-4143-9107-d6229ec9cdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:38.299671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc3779-cf11-4143-9107-d6229ec9cdc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:38.300314 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc3779-cf11-4143-9107-d6229ec9cdc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:38.302268 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc3779-cf11-4143-9107-d6229ec9cdc3 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:38.334741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1383/2748] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:38 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:38.338881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed7bf0-ed83-4aed-b651-75477dfd83c4 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:38.349436 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed7bf0-ed83-4aed-b651-75477dfd83c4 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:38.409711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d5b0a-9ed0-460d-a839-6328bd83360d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:38.410425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d5b0a-9ed0-460d-a839-6328bd83360d admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:38.410752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d5b0a-9ed0-460d-a839-6328bd83360d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:38.411089 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d5b0a-9ed0-460d-a839-6328bd83360d admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:38.411723 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d5b0a-9ed0-460d-a839-6328bd83360d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:38.413709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d5b0a-9ed0-460d-a839-6328bd83360d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:38.439091 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1366/2749] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:38 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:38.440287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc3779-cf11-4143-9107-d6229ec9cdc3 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:38.451728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc3779-cf11-4143-9107-d6229ec9cdc3 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:38.457619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea35f0d-086c-415e-b4c9-5a8b23e6c4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:38.458285 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea35f0d-086c-415e-b4c9-5a8b23e6c4f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:38.458563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea35f0d-086c-415e-b4c9-5a8b23e6c4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:38.458836 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea35f0d-086c-415e-b4c9-5a8b23e6c4f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:38.459482 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea35f0d-086c-415e-b4c9-5a8b23e6c4f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:38.461109 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea35f0d-086c-415e-b4c9-5a8b23e6c4f3 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:38.471859 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1384/2750] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:38.476034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d5b0a-9ed0-460d-a839-6328bd83360d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:38.487370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d5b0a-9ed0-460d-a839-6328bd83360d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:38.492801 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c69a1-2bee-43c9-b89f-570271a6959d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:38.493494 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c69a1-2bee-43c9-b89f-570271a6959d admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:38.493899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c69a1-2bee-43c9-b89f-570271a6959d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:38.494063 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c69a1-2bee-43c9-b89f-570271a6959d admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:38.494817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c69a1-2bee-43c9-b89f-570271a6959d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:38.496653 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c69a1-2bee-43c9-b89f-570271a6959d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:38.507600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1367/2751] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:38.512012 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea35f0d-086c-415e-b4c9-5a8b23e6c4f3 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:38.523258 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea35f0d-086c-415e-b4c9-5a8b23e6c4f3 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:38.529648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d82d8-d6e9-4fb7-9c5f-07b86cb6b730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:38.530606 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d82d8-d6e9-4fb7-9c5f-07b86cb6b730 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:38.530961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d82d8-d6e9-4fb7-9c5f-07b86cb6b730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:38.531284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d82d8-d6e9-4fb7-9c5f-07b86cb6b730 admin admin] PATH_INFO: `/v3/projects/ab1c21ee432649b6a06ddb5a02554d7e/users/6639e864e7164cdf82a45fe62189f3c2/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:38.547133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d82d8-d6e9-4fb7-9c5f-07b86cb6b730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab1c21ee432649b6a06ddb5a02554d7e, user_id=6639e864e7164cdf82a45fe62189f3c2, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:38.549837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d82d8-d6e9-4fb7-9c5f-07b86cb6b730 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:38.565771 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1385/2752] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:38 2022] PUT /identity/v3/projects/ab1c21ee432649b6a06ddb5a02554d7e/users/6639e864e7164cdf82a45fe62189f3c2/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:38.577988 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a88800-b951-416b-90c1-367c86d5e06e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:38.578246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a88800-b951-416b-90c1-367c86d5e06e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:38.578546 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a88800-b951-416b-90c1-367c86d5e06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:38.610767 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c8a88800-b951-416b-90c1-367c86d5e06e None None] MFA Rules not processed for user `6639e864e7164cdf82a45fe62189f3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:38.611965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c8a88800-b951-416b-90c1-367c86d5e06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:38.681243 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1368/2753] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:38.686010 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d82d8-d6e9-4fb7-9c5f-07b86cb6b730 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:38.728520 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:38.763755 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d82d8-d6e9-4fb7-9c5f-07b86cb6b730 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:38.770145 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013bc47-f845-4746-9114-c815aae4e017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:38.770759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013bc47-f845-4746-9114-c815aae4e017 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:38.771063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013bc47-f845-4746-9114-c815aae4e017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:38.771334 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013bc47-f845-4746-9114-c815aae4e017 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:38.771981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013bc47-f845-4746-9114-c815aae4e017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:38.773926 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013bc47-f845-4746-9114-c815aae4e017 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:38.805719 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1386/2754] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:38 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:38.810135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a88800-b951-416b-90c1-367c86d5e06e None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:38.821673 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a88800-b951-416b-90c1-367c86d5e06e None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:38.861784 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:38.886291 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25033fbf-473b-4eb2-b475-a43c8fa5a5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:38.886906 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25033fbf-473b-4eb2-b475-a43c8fa5a5e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:38.887181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25033fbf-473b-4eb2-b475-a43c8fa5a5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:38.887453 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25033fbf-473b-4eb2-b475-a43c8fa5a5e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:38.888135 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25033fbf-473b-4eb2-b475-a43c8fa5a5e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:38.890142 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25033fbf-473b-4eb2-b475-a43c8fa5a5e6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:38.912844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1369/2755] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:38 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:38.916961 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013bc47-f845-4746-9114-c815aae4e017 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:38.927542 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013bc47-f845-4746-9114-c815aae4e017 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:38.932569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0d05c-0511-4158-8045-8e17ddb6129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:38.933181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0d05c-0511-4158-8045-8e17ddb6129f admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:38.933468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0d05c-0511-4158-8045-8e17ddb6129f admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:38.933739 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0d05c-0511-4158-8045-8e17ddb6129f admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:38.934382 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0d05c-0511-4158-8045-8e17ddb6129f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:38.935976 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0d05c-0511-4158-8045-8e17ddb6129f admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:38.946678 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1387/2756] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:38.951001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25033fbf-473b-4eb2-b475-a43c8fa5a5e6 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:38.961707 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25033fbf-473b-4eb2-b475-a43c8fa5a5e6 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:38.966599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1a3ad-450a-48bf-88bc-90428a548344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:38.967254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1a3ad-450a-48bf-88bc-90428a548344 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:38.967535 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1a3ad-450a-48bf-88bc-90428a548344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:38.967814 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1a3ad-450a-48bf-88bc-90428a548344 admin admin] PATH_INFO: `/v3/projects/6b754edd769145ada6058ba697c3a5bd/users/b2eeae20051d451faf89ef03d783ebb7/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:38.983052 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1a3ad-450a-48bf-88bc-90428a548344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b754edd769145ada6058ba697c3a5bd, user_id=b2eeae20051d451faf89ef03d783ebb7, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:38.985743 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1a3ad-450a-48bf-88bc-90428a548344 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:39.001067 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1370/2757] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:38 2022] PUT /identity/v3/projects/6b754edd769145ada6058ba697c3a5bd/users/b2eeae20051d451faf89ef03d783ebb7/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:39.005233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0d05c-0511-4158-8045-8e17ddb6129f admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:39.015451 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0d05c-0511-4158-8045-8e17ddb6129f admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:39.086523 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e45a17-ac33-4678-8b4d-f1ca701b041e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:39.087146 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e45a17-ac33-4678-8b4d-f1ca701b041e admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:39.087421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e45a17-ac33-4678-8b4d-f1ca701b041e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:39.087790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e45a17-ac33-4678-8b4d-f1ca701b041e admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:39.088313 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e45a17-ac33-4678-8b4d-f1ca701b041e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:39.090039 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e45a17-ac33-4678-8b4d-f1ca701b041e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:39.100086 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1388/2758] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:39.104474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1a3ad-450a-48bf-88bc-90428a548344 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:39.117105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1a3ad-450a-48bf-88bc-90428a548344 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:39.122080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3481c9-6b70-4fe5-a241-9408c0b6657e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:39.122751 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3481c9-6b70-4fe5-a241-9408c0b6657e admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:39.123085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3481c9-6b70-4fe5-a241-9408c0b6657e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:39.123406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3481c9-6b70-4fe5-a241-9408c0b6657e admin admin] PATH_INFO: `/v3/projects/6b754edd769145ada6058ba697c3a5bd/users/b2eeae20051d451faf89ef03d783ebb7/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:39.126572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3481c9-6b70-4fe5-a241-9408c0b6657e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b754edd769145ada6058ba697c3a5bd, user_id=b2eeae20051d451faf89ef03d783ebb7, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:39.129642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3481c9-6b70-4fe5-a241-9408c0b6657e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:39.140127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1371/2759] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:39 2022] PUT /identity/v3/projects/6b754edd769145ada6058ba697c3a5bd/users/b2eeae20051d451faf89ef03d783ebb7/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:39.150757 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2878c2-1fd2-4ebe-b42d-a1e18fbd4f43 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:39.151093 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2878c2-1fd2-4ebe-b42d-a1e18fbd4f43 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:39.151400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2878c2-1fd2-4ebe-b42d-a1e18fbd4f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:39.183271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-0e2878c2-1fd2-4ebe-b42d-a1e18fbd4f43 None None] MFA Rules not processed for user `b2eeae20051d451faf89ef03d783ebb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:39.185345 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-0e2878c2-1fd2-4ebe-b42d-a1e18fbd4f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:39.267838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1389/2760] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:39.280265 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3481c9-6b70-4fe5-a241-9408c0b6657e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:39.282029 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7e3481c9-6b70-4fe5-a241-9408c0b6657e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:39.340836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3481c9-6b70-4fe5-a241-9408c0b6657e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:39.344657 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f27088-6f6b-4563-9331-cb2d5ee671d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:39.345313 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f27088-6f6b-4563-9331-cb2d5ee671d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:39.345587 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f27088-6f6b-4563-9331-cb2d5ee671d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:39.345855 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f27088-6f6b-4563-9331-cb2d5ee671d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:39.356645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f27088-6f6b-4563-9331-cb2d5ee671d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:39.358201 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f27088-6f6b-4563-9331-cb2d5ee671d5 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:39.368004 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1372/2761] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:39.440827 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2878c2-1fd2-4ebe-b42d-a1e18fbd4f43 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:39.451317 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2878c2-1fd2-4ebe-b42d-a1e18fbd4f43 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:39.455977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8d363-2eb0-4023-b44a-0bee9438d223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:39.456587 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8d363-2eb0-4023-b44a-0bee9438d223 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:39.456865 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8d363-2eb0-4023-b44a-0bee9438d223 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:39.457163 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8d363-2eb0-4023-b44a-0bee9438d223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:39.489754 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:39.520770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8d363-2eb0-4023-b44a-0bee9438d223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:39.522375 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b8d363-2eb0-4023-b44a-0bee9438d223 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:39.533522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1390/2762] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:39.846342 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f27088-6f6b-4563-9331-cb2d5ee671d5 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:39.915460 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f27088-6f6b-4563-9331-cb2d5ee671d5 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:39.922903 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acc262-562e-4d54-9a91-aef9cdaf5309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:39.923800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acc262-562e-4d54-9a91-aef9cdaf5309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:39.924138 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acc262-562e-4d54-9a91-aef9cdaf5309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:39.924502 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acc262-562e-4d54-9a91-aef9cdaf5309 admin admin] PATH_INFO: `/v3/users/6639e864e7164cdf82a45fe62189f3c2` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:39.926369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84acc262-562e-4d54-9a91-aef9cdaf5309 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6639e864e7164cdf82a45fe62189f3c2)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:39.929360 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84acc262-562e-4d54-9a91-aef9cdaf5309 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:39.972794 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:39.985118 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-84acc262-562e-4d54-9a91-aef9cdaf5309 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6639e864e7164cdf82a45fe62189f3c2'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:39.994886 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-84acc262-562e-4d54-9a91-aef9cdaf5309 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6639e864e7164cdf82a45fe62189f3c2'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:39.995712 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-84acc262-562e-4d54-9a91-aef9cdaf5309 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6639e864e7164cdf82a45fe62189f3c2'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:39.996422 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-84acc262-562e-4d54-9a91-aef9cdaf5309 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6639e864e7164cdf82a45fe62189f3c2'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:40.009994 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-84acc262-562e-4d54-9a91-aef9cdaf5309 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6639e864e7164cdf82a45fe62189f3c2'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:40.022500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1373/2763] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:39 2022] DELETE /identity/v3/users/6639e864e7164cdf82a45fe62189f3c2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:40.027210 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671d103-fb73-4b4d-87fa-fea9ca9a2131 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:40.027494 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671d103-fb73-4b4d-87fa-fea9ca9a2131 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:40.027761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671d103-fb73-4b4d-87fa-fea9ca9a2131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:40.046542 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d671d103-fb73-4b4d-87fa-fea9ca9a2131 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:40.047741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d671d103-fb73-4b4d-87fa-fea9ca9a2131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:40.111316 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1391/2764] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:40.122864 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acc262-562e-4d54-9a91-aef9cdaf5309 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:40.124546 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-84acc262-562e-4d54-9a91-aef9cdaf5309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:40.190248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acc262-562e-4d54-9a91-aef9cdaf5309 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:40.194656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e4a0e-7fa8-43b3-932a-45112e023cb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:40.195278 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e4a0e-7fa8-43b3-932a-45112e023cb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:40.195559 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e4a0e-7fa8-43b3-932a-45112e023cb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:40.195833 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e4a0e-7fa8-43b3-932a-45112e023cb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:40.206802 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e4a0e-7fa8-43b3-932a-45112e023cb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:40.208348 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e4a0e-7fa8-43b3-932a-45112e023cb9 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:40.218542 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1374/2765] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:56:40.569269 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671d103-fb73-4b4d-87fa-fea9ca9a2131 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:40.571470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d671d103-fb73-4b4d-87fa-fea9ca9a2131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:40.585235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671d103-fb73-4b4d-87fa-fea9ca9a2131 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:40.597365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3c108-49ea-4895-a194-156eeb019bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:40.598024 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3c108-49ea-4895-a194-156eeb019bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:40.598421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3c108-49ea-4895-a194-156eeb019bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:40.598690 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3c108-49ea-4895-a194-156eeb019bab admin admin] PATH_INFO: `/v3/projects/ab1c21ee432649b6a06ddb5a02554d7e` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:40.599838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3c108-49ea-4895-a194-156eeb019bab admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab1c21ee432649b6a06ddb5a02554d7e)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:40.601838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3c108-49ea-4895-a194-156eeb019bab admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:40.649876 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-3aa3c108-49ea-4895-a194-156eeb019bab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab1c21ee432649b6a06ddb5a02554d7e', 'request_id': 'req-3aa3c108-49ea-4895-a194-156eeb019bab'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:40.656941 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-3aa3c108-49ea-4895-a194-156eeb019bab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab1c21ee432649b6a06ddb5a02554d7e', 'request_id': 'req-3aa3c108-49ea-4895-a194-156eeb019bab'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:40.664838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-3aa3c108-49ea-4895-a194-156eeb019bab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab1c21ee432649b6a06ddb5a02554d7e', 'request_id': 'req-3aa3c108-49ea-4895-a194-156eeb019bab'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:40.676213 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-3aa3c108-49ea-4895-a194-156eeb019bab admin admin] The token cache is being invalidate because project ab1c21ee432649b6a06ddb5a02554d7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:40.676536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-3aa3c108-49ea-4895-a194-156eeb019bab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:40.677320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-3aa3c108-49ea-4895-a194-156eeb019bab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:40.678879 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1392/2766] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:40 2022] DELETE /identity/v3/projects/ab1c21ee432649b6a06ddb5a02554d7e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:40.683226 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e4a0e-7fa8-43b3-932a-45112e023cb9 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:40.693332 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e4a0e-7fa8-43b3-932a-45112e023cb9 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:40.756138 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8d4a0-fed4-4333-a5ef-c91fee622fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:40.756896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8d4a0-fed4-4333-a5ef-c91fee622fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:40.757211 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8d4a0-fed4-4333-a5ef-c91fee622fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:40.757738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8d4a0-fed4-4333-a5ef-c91fee622fbf admin admin] PATH_INFO: `/v3/users/b2eeae20051d451faf89ef03d783ebb7` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:40.759046 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f8d4a0-fed4-4333-a5ef-c91fee622fbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2eeae20051d451faf89ef03d783ebb7)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:40.761100 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f8d4a0-fed4-4333-a5ef-c91fee622fbf admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:40.813053 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-01f8d4a0-fed4-4333-a5ef-c91fee622fbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2eeae20051d451faf89ef03d783ebb7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:40.821423 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-01f8d4a0-fed4-4333-a5ef-c91fee622fbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2eeae20051d451faf89ef03d783ebb7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:40.822146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-01f8d4a0-fed4-4333-a5ef-c91fee622fbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2eeae20051d451faf89ef03d783ebb7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:40.822765 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-01f8d4a0-fed4-4333-a5ef-c91fee622fbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2eeae20051d451faf89ef03d783ebb7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:40.824834 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:40.848457 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-01f8d4a0-fed4-4333-a5ef-c91fee622fbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2eeae20051d451faf89ef03d783ebb7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:40.869370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1375/2767] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:40 2022] DELETE /identity/v3/users/b2eeae20051d451faf89ef03d783ebb7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:41.160436 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3c108-49ea-4895-a194-156eeb019bab admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:41.161981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-3aa3c108-49ea-4895-a194-156eeb019bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:41.175734 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3c108-49ea-4895-a194-156eeb019bab admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:41.228101 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:41.239259 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8b48da-3515-4bc5-b239-932839a307cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:41.240078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b48da-3515-4bc5-b239-932839a307cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:41.240517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b48da-3515-4bc5-b239-932839a307cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:41.240908 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b48da-3515-4bc5-b239-932839a307cd admin admin] PATH_INFO: `/v3/projects/6b754edd769145ada6058ba697c3a5bd` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:41.242240 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8b48da-3515-4bc5-b239-932839a307cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b754edd769145ada6058ba697c3a5bd)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:41.244127 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8b48da-3515-4bc5-b239-932839a307cd admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:41.294800 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1f8b48da-3515-4bc5-b239-932839a307cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b754edd769145ada6058ba697c3a5bd', 'request_id': 'req-1f8b48da-3515-4bc5-b239-932839a307cd'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:41.301759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1f8b48da-3515-4bc5-b239-932839a307cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b754edd769145ada6058ba697c3a5bd', 'request_id': 'req-1f8b48da-3515-4bc5-b239-932839a307cd'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:41.312446 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1f8b48da-3515-4bc5-b239-932839a307cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b754edd769145ada6058ba697c3a5bd', 'request_id': 'req-1f8b48da-3515-4bc5-b239-932839a307cd'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:41.324227 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1f8b48da-3515-4bc5-b239-932839a307cd admin admin] The token cache is being invalidate because project 6b754edd769145ada6058ba697c3a5bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:41.324521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1f8b48da-3515-4bc5-b239-932839a307cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:41.325194 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-1f8b48da-3515-4bc5-b239-932839a307cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:41.326717 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1393/2768] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:41 2022] DELETE /identity/v3/projects/6b754edd769145ada6058ba697c3a5bd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:41.332464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18245c9a-4dbb-4e5b-8ae0-b68409c10fcb None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:41.332735 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18245c9a-4dbb-4e5b-8ae0-b68409c10fcb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:41.333009 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18245c9a-4dbb-4e5b-8ae0-b68409c10fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:41.354560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-18245c9a-4dbb-4e5b-8ae0-b68409c10fcb None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:41.355825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-18245c9a-4dbb-4e5b-8ae0-b68409c10fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:41.426107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1376/2769] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:41.435993 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a210a-bdc4-47cc-8c44-35aca800f03b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:41.436224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a210a-bdc4-47cc-8c44-35aca800f03b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:41.436492 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a210a-bdc4-47cc-8c44-35aca800f03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:41.457920 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d65a210a-bdc4-47cc-8c44-35aca800f03b None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:41.459202 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d65a210a-bdc4-47cc-8c44-35aca800f03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:41.468854 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1394/2770] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:41.473958 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18245c9a-4dbb-4e5b-8ae0-b68409c10fcb None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:41.484978 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18245c9a-4dbb-4e5b-8ae0-b68409c10fcb None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:41.490491 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7be98-380a-4f0e-9c75-df596b3b67ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:41.491106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7be98-380a-4f0e-9c75-df596b3b67ca admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:41.491382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7be98-380a-4f0e-9c75-df596b3b67ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:41.491651 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae7be98-380a-4f0e-9c75-df596b3b67ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:41.492305 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7be98-380a-4f0e-9c75-df596b3b67ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:56:41.492615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7be98-380a-4f0e-9c75-df596b3b67ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:41.494257 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae7be98-380a-4f0e-9c75-df596b3b67ca admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:41.507040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1377/2771] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:56:41.513230 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a210a-bdc4-47cc-8c44-35aca800f03b None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:41.523071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a210a-bdc4-47cc-8c44-35aca800f03b None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:41.529305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e7a48-16aa-41a1-b845-26541e9eb6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:41.530029 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e7a48-16aa-41a1-b845-26541e9eb6d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:41.530332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e7a48-16aa-41a1-b845-26541e9eb6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:41.530615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e7a48-16aa-41a1-b845-26541e9eb6d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:41.531352 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e7a48-16aa-41a1-b845-26541e9eb6d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:41.533473 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e7a48-16aa-41a1-b845-26541e9eb6d2 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:41.565679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1395/2772] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:41 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:41.571248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7be98-380a-4f0e-9c75-df596b3b67ca admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:41.583535 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae7be98-380a-4f0e-9c75-df596b3b67ca admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:41.648595 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bed7bc-de95-4a70-8a1a-ad3182842047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:41.649256 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bed7bc-de95-4a70-8a1a-ad3182842047 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:41.649598 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bed7bc-de95-4a70-8a1a-ad3182842047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:41.649868 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bed7bc-de95-4a70-8a1a-ad3182842047 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:41.650507 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bed7bc-de95-4a70-8a1a-ad3182842047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:41.652508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bed7bc-de95-4a70-8a1a-ad3182842047 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:41.680847 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1378/2773] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:41 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:41.682095 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e7a48-16aa-41a1-b845-26541e9eb6d2 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:41.694143 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e7a48-16aa-41a1-b845-26541e9eb6d2 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:41.699827 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6c0ea-4954-428c-b530-dee2b8342346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:41.700456 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6c0ea-4954-428c-b530-dee2b8342346 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:41.700727 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6c0ea-4954-428c-b530-dee2b8342346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:41.700994 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6c0ea-4954-428c-b530-dee2b8342346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:41.701836 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6c0ea-4954-428c-b530-dee2b8342346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:41.703554 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6c0ea-4954-428c-b530-dee2b8342346 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:41.719382 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1396/2774] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:41.724006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bed7bc-de95-4a70-8a1a-ad3182842047 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:41.734331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bed7bc-de95-4a70-8a1a-ad3182842047 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:41.739295 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917c2f6-e5e0-4bb7-a070-881f0ccef325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:41.740232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917c2f6-e5e0-4bb7-a070-881f0ccef325 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:41.740232 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917c2f6-e5e0-4bb7-a070-881f0ccef325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:41.740474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917c2f6-e5e0-4bb7-a070-881f0ccef325 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:41.741123 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917c2f6-e5e0-4bb7-a070-881f0ccef325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:41.742714 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917c2f6-e5e0-4bb7-a070-881f0ccef325 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:41.761543 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1379/2775] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:41.763352 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6c0ea-4954-428c-b530-dee2b8342346 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:41.774219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6c0ea-4954-428c-b530-dee2b8342346 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:41.778797 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e832ee-2841-439e-bbbc-f05ef0cac012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:41.779504 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e832ee-2841-439e-bbbc-f05ef0cac012 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:41.779812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e832ee-2841-439e-bbbc-f05ef0cac012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:41.780083 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e832ee-2841-439e-bbbc-f05ef0cac012 admin admin] PATH_INFO: `/v3/projects/abb459a3fade4a088724d66d43dd1067/users/c9fa90d0142145b79eb6ee5b3926b9a9/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:41.797255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e832ee-2841-439e-bbbc-f05ef0cac012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb459a3fade4a088724d66d43dd1067, user_id=c9fa90d0142145b79eb6ee5b3926b9a9, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:41.799914 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e832ee-2841-439e-bbbc-f05ef0cac012 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:41.816369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1397/2776] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:41 2022] PUT /identity/v3/projects/abb459a3fade4a088724d66d43dd1067/users/c9fa90d0142145b79eb6ee5b3926b9a9/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:41.826966 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd499777-17b8-4d7b-8884-a40eef355fa5 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:41.827246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd499777-17b8-4d7b-8884-a40eef355fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:41.827569 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd499777-17b8-4d7b-8884-a40eef355fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:41.856953 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-dd499777-17b8-4d7b-8884-a40eef355fa5 None None] MFA Rules not processed for user `c9fa90d0142145b79eb6ee5b3926b9a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:41.858230 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-dd499777-17b8-4d7b-8884-a40eef355fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:41.888338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:41.935524 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1380/2777] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:41.940302 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e832ee-2841-439e-bbbc-f05ef0cac012 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:42.003224 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e832ee-2841-439e-bbbc-f05ef0cac012 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:42.008105 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd39e2-479d-40be-9133-4ccbcb880eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:42.008702 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd39e2-479d-40be-9133-4ccbcb880eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:42.008979 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd39e2-479d-40be-9133-4ccbcb880eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:42.009266 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd39e2-479d-40be-9133-4ccbcb880eb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:42.009911 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd39e2-479d-40be-9133-4ccbcb880eb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:42.011912 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd39e2-479d-40be-9133-4ccbcb880eb6 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:42.040280 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1398/2778] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:41 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:42.044401 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd499777-17b8-4d7b-8884-a40eef355fa5 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:42.055555 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd499777-17b8-4d7b-8884-a40eef355fa5 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:42.117590 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b972f2c-d1ed-4b34-9dc8-963da1dda5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:42.118212 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b972f2c-d1ed-4b34-9dc8-963da1dda5af admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:42.118488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b972f2c-d1ed-4b34-9dc8-963da1dda5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:42.118755 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b972f2c-d1ed-4b34-9dc8-963da1dda5af admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:42.119403 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b972f2c-d1ed-4b34-9dc8-963da1dda5af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:42.121431 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b972f2c-d1ed-4b34-9dc8-963da1dda5af admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:42.144812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1381/2779] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:42 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:42.148915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd39e2-479d-40be-9133-4ccbcb880eb6 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:42.151479 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b972f2c-d1ed-4b34-9dc8-963da1dda5af admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:42.153283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4b972f2c-d1ed-4b34-9dc8-963da1dda5af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:42.156325 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:42.160369 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd39e2-479d-40be-9133-4ccbcb880eb6 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:42.166951 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a6226-87f7-400b-b6fd-2425ad7b7bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:42.167629 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a6226-87f7-400b-b6fd-2425ad7b7bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:42.168094 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a6226-87f7-400b-b6fd-2425ad7b7bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:42.168419 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a6226-87f7-400b-b6fd-2425ad7b7bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:42.169200 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a6226-87f7-400b-b6fd-2425ad7b7bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:42.171021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a6226-87f7-400b-b6fd-2425ad7b7bc6 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:42.187100 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1399/2780] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:42.191463 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a6226-87f7-400b-b6fd-2425ad7b7bc6 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:42.208478 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a6226-87f7-400b-b6fd-2425ad7b7bc6 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:42.218831 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba8581-2ad8-4601-9a28-32459f15c7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:42.218831 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba8581-2ad8-4601-9a28-32459f15c7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:42.218831 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba8581-2ad8-4601-9a28-32459f15c7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:42.218831 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba8581-2ad8-4601-9a28-32459f15c7d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:42.219331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba8581-2ad8-4601-9a28-32459f15c7d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:42.219331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba8581-2ad8-4601-9a28-32459f15c7d7 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:42.219513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b972f2c-d1ed-4b34-9dc8-963da1dda5af admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:42.223935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c7284-2eeb-468b-a0fa-6afb909896a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:42.224551 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c7284-2eeb-468b-a0fa-6afb909896a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:42.224809 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c7284-2eeb-468b-a0fa-6afb909896a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:42.225103 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c7284-2eeb-468b-a0fa-6afb909896a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:42.227076 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1e8c7284-2eeb-468b-a0fa-6afb909896a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:42.228594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1400/2781] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:42.232934 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba8581-2ad8-4601-9a28-32459f15c7d7 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:42.243536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba8581-2ad8-4601-9a28-32459f15c7d7 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:42.248525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caeb001-1a06-4828-84ae-40e5f0fba5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:42.249186 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caeb001-1a06-4828-84ae-40e5f0fba5ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:42.249521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caeb001-1a06-4828-84ae-40e5f0fba5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:42.249774 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caeb001-1a06-4828-84ae-40e5f0fba5ea admin admin] PATH_INFO: `/v3/projects/cc464062efa544128155fe6cb226ba3b/users/b278dc1130054323a250ab8af3042fd3/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:42.264833 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caeb001-1a06-4828-84ae-40e5f0fba5ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc464062efa544128155fe6cb226ba3b, user_id=b278dc1130054323a250ab8af3042fd3, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:42.267632 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caeb001-1a06-4828-84ae-40e5f0fba5ea admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:42.281189 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1401/2782] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:42 2022] PUT /identity/v3/projects/cc464062efa544128155fe6cb226ba3b/users/b278dc1130054323a250ab8af3042fd3/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:42.292505 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc865c4-cdbd-424c-a8ec-d76a61355446 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:42.292779 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc865c4-cdbd-424c-a8ec-d76a61355446 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:42.293230 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc865c4-cdbd-424c-a8ec-d76a61355446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:42.302732 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c7284-2eeb-468b-a0fa-6afb909896a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:42.304173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c7284-2eeb-468b-a0fa-6afb909896a9 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:42.313506 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1382/2783] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:56:42.321832 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-fbc865c4-cdbd-424c-a8ec-d76a61355446 None None] MFA Rules not processed for user `b278dc1130054323a250ab8af3042fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:42.323318 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-fbc865c4-cdbd-424c-a8ec-d76a61355446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:42.390660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1402/2784] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:42.395814 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c7284-2eeb-468b-a0fa-6afb909896a9 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:42.407937 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c7284-2eeb-468b-a0fa-6afb909896a9 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:42.418744 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:42.478191 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca94ee-49cc-4f9f-8e60-5edebc497c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:42.478490 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca94ee-49cc-4f9f-8e60-5edebc497c68 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:42.478916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca94ee-49cc-4f9f-8e60-5edebc497c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:42.479203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca94ee-49cc-4f9f-8e60-5edebc497c68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:42.480084 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca94ee-49cc-4f9f-8e60-5edebc497c68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:42.482051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca94ee-49cc-4f9f-8e60-5edebc497c68 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:42.514058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1383/2785] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:42 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:42.515231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc865c4-cdbd-424c-a8ec-d76a61355446 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:42.528080 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc865c4-cdbd-424c-a8ec-d76a61355446 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:42.591534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644a2be-d2b5-40cd-b5d5-27689fb72836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:42.592160 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644a2be-d2b5-40cd-b5d5-27689fb72836 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:42.592441 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644a2be-d2b5-40cd-b5d5-27689fb72836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:42.592812 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644a2be-d2b5-40cd-b5d5-27689fb72836 admin admin] PATH_INFO: `/v3/users` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:42.593386 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644a2be-d2b5-40cd-b5d5-27689fb72836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:42.595373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644a2be-d2b5-40cd-b5d5-27689fb72836 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:42.616423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1403/2786] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:42 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:42.621295 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca94ee-49cc-4f9f-8e60-5edebc497c68 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:42.632273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca94ee-49cc-4f9f-8e60-5edebc497c68 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:42.638121 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6669ab7-cc91-4234-9ee0-e07d48cf0673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:42.638783 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6669ab7-cc91-4234-9ee0-e07d48cf0673 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:42.639061 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6669ab7-cc91-4234-9ee0-e07d48cf0673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:42.639370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6669ab7-cc91-4234-9ee0-e07d48cf0673 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:42.640140 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6669ab7-cc91-4234-9ee0-e07d48cf0673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:42.641825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6669ab7-cc91-4234-9ee0-e07d48cf0673 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:42.653500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1384/2787] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:42.658181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644a2be-d2b5-40cd-b5d5-27689fb72836 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:42.670492 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644a2be-d2b5-40cd-b5d5-27689fb72836 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:42.675366 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a54b31-5371-428e-90fe-e6a94561978a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:42.676017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a54b31-5371-428e-90fe-e6a94561978a admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:42.676295 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a54b31-5371-428e-90fe-e6a94561978a admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:42.676573 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a54b31-5371-428e-90fe-e6a94561978a admin admin] PATH_INFO: `/v3/projects/fd76c42ed5da43b58cce0feddbfee27e/users/ee002959335f4cb788b9a3d5a7f99ba8/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:42.693875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a54b31-5371-428e-90fe-e6a94561978a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd76c42ed5da43b58cce0feddbfee27e, user_id=ee002959335f4cb788b9a3d5a7f99ba8, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:42.697184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a54b31-5371-428e-90fe-e6a94561978a admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:42.716661 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1404/2788] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:42 2022] PUT /identity/v3/projects/fd76c42ed5da43b58cce0feddbfee27e/users/ee002959335f4cb788b9a3d5a7f99ba8/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:42.720846 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6669ab7-cc91-4234-9ee0-e07d48cf0673 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:42.732452 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6669ab7-cc91-4234-9ee0-e07d48cf0673 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:42.815453 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860f581-54f6-496b-9690-6e198a5ce787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:42.816433 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860f581-54f6-496b-9690-6e198a5ce787 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:42.816713 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860f581-54f6-496b-9690-6e198a5ce787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:42.817175 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860f581-54f6-496b-9690-6e198a5ce787 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:42.817826 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e860f581-54f6-496b-9690-6e198a5ce787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:42.819431 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e860f581-54f6-496b-9690-6e198a5ce787 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:42.833187 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1385/2789] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:42.835646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a54b31-5371-428e-90fe-e6a94561978a admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:42.850097 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a54b31-5371-428e-90fe-e6a94561978a admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:42.855270 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c5c37-38cd-4142-b45c-888f628aa15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:42.856016 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c5c37-38cd-4142-b45c-888f628aa15f admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:42.856368 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c5c37-38cd-4142-b45c-888f628aa15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:42.856716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c5c37-38cd-4142-b45c-888f628aa15f admin admin] PATH_INFO: `/v3/projects/fd76c42ed5da43b58cce0feddbfee27e/users/ee002959335f4cb788b9a3d5a7f99ba8/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:42.859264 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c5c37-38cd-4142-b45c-888f628aa15f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd76c42ed5da43b58cce0feddbfee27e, user_id=ee002959335f4cb788b9a3d5a7f99ba8, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:42.862103 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c5c37-38cd-4142-b45c-888f628aa15f admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:42.869760 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:42.876903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1405/2790] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:42 2022] PUT /identity/v3/projects/fd76c42ed5da43b58cce0feddbfee27e/users/ee002959335f4cb788b9a3d5a7f99ba8/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:42.889241 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5c55e-e14d-41aa-b756-7e08c8d17344 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:42.889711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5c55e-e14d-41aa-b756-7e08c8d17344 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:42.889838 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5c55e-e14d-41aa-b756-7e08c8d17344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:42.921230 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-29d5c55e-e14d-41aa-b756-7e08c8d17344 None None] MFA Rules not processed for user `ee002959335f4cb788b9a3d5a7f99ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:42.922938 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-29d5c55e-e14d-41aa-b756-7e08c8d17344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:43.004089 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1386/2791] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:43.016757 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c5c37-38cd-4142-b45c-888f628aa15f admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:43.083130 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c5c37-38cd-4142-b45c-888f628aa15f admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:43.087269 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef54d74-22c4-4ba0-84ff-8ec2da4dbbc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:43.088063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef54d74-22c4-4ba0-84ff-8ec2da4dbbc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:43.088784 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef54d74-22c4-4ba0-84ff-8ec2da4dbbc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:43.089114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef54d74-22c4-4ba0-84ff-8ec2da4dbbc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:43.152899 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef54d74-22c4-4ba0-84ff-8ec2da4dbbc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:43.154402 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef54d74-22c4-4ba0-84ff-8ec2da4dbbc1 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:43.163791 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1406/2792] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:44.542090 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5c55e-e14d-41aa-b756-7e08c8d17344 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:44.543604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-29d5c55e-e14d-41aa-b756-7e08c8d17344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:44.554559 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:44.617109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5c55e-e14d-41aa-b756-7e08c8d17344 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:44.621872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46547a-96fb-48e8-a051-23f89140e51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:44.622585 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46547a-96fb-48e8-a051-23f89140e51d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:44.622884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46547a-96fb-48e8-a051-23f89140e51d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:44.623177 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46547a-96fb-48e8-a051-23f89140e51d admin admin] PATH_INFO: `/v3/users/8d82b4df4e7c49b38aac207507be6f1e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:44.624443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46547a-96fb-48e8-a051-23f89140e51d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d82b4df4e7c49b38aac207507be6f1e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:44.626603 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46547a-96fb-48e8-a051-23f89140e51d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:44.686964 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3e46547a-96fb-48e8-a051-23f89140e51d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d82b4df4e7c49b38aac207507be6f1e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:44.696602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3e46547a-96fb-48e8-a051-23f89140e51d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d82b4df4e7c49b38aac207507be6f1e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:44.697358 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3e46547a-96fb-48e8-a051-23f89140e51d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d82b4df4e7c49b38aac207507be6f1e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:44.698043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3e46547a-96fb-48e8-a051-23f89140e51d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d82b4df4e7c49b38aac207507be6f1e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:44.711099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3e46547a-96fb-48e8-a051-23f89140e51d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d82b4df4e7c49b38aac207507be6f1e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:44.723621 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1387/2793] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:44 2022] DELETE /identity/v3/users/8d82b4df4e7c49b38aac207507be6f1e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:44.728525 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e2693-72b5-454a-81c2-3cc09ca88a21 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:44.729044 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e2693-72b5-454a-81c2-3cc09ca88a21 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:44.729174 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e2693-72b5-454a-81c2-3cc09ca88a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:44.737627 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46547a-96fb-48e8-a051-23f89140e51d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:44.739052 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3e46547a-96fb-48e8-a051-23f89140e51d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:44.751808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-589e2693-72b5-454a-81c2-3cc09ca88a21 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:44.751808 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-589e2693-72b5-454a-81c2-3cc09ca88a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:44.801302 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46547a-96fb-48e8-a051-23f89140e51d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:44.805758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48cf57-49e8-447e-b8a2-c1fd8659a9e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:44.806380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48cf57-49e8-447e-b8a2-c1fd8659a9e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:44.806659 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48cf57-49e8-447e-b8a2-c1fd8659a9e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:44.806929 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48cf57-49e8-447e-b8a2-c1fd8659a9e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:44.808963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ed48cf57-49e8-447e-b8a2-c1fd8659a9e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:44.820855 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1407/2794] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:44.831436 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e2693-72b5-454a-81c2-3cc09ca88a21 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:44.833416 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-589e2693-72b5-454a-81c2-3cc09ca88a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:44.869045 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:44.884532 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48cf57-49e8-447e-b8a2-c1fd8659a9e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:44.886291 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48cf57-49e8-447e-b8a2-c1fd8659a9e7 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:44.895865 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1388/2795] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:56:44.903773 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e2693-72b5-454a-81c2-3cc09ca88a21 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:44.908168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e3015-cd74-4283-b983-8a6196f1560d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:44.908780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e3015-cd74-4283-b983-8a6196f1560d service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:44.909054 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e3015-cd74-4283-b983-8a6196f1560d service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:44.909346 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e3015-cd74-4283-b983-8a6196f1560d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:44.919565 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e3015-cd74-4283-b983-8a6196f1560d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:44.920979 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e3015-cd74-4283-b983-8a6196f1560d service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:44.925077 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=100 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:44.932669 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1408/2796] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:56:45.236217 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48cf57-49e8-447e-b8a2-c1fd8659a9e7 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:45.237899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ed48cf57-49e8-447e-b8a2-c1fd8659a9e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:45.254985 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48cf57-49e8-447e-b8a2-c1fd8659a9e7 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:45.261984 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:45.262715 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:45.262908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:45.263262 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc admin admin] PATH_INFO: `/v3/projects/7abad03e64cb47fba3d37d5f84b3c899` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:45.264312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7abad03e64cb47fba3d37d5f84b3c899)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:45.266255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:45.323007 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7abad03e64cb47fba3d37d5f84b3c899', 'request_id': 'req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:45.332617 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7abad03e64cb47fba3d37d5f84b3c899', 'request_id': 'req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:45.343471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7abad03e64cb47fba3d37d5f84b3c899', 'request_id': 'req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:45.354293 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc admin admin] The token cache is being invalidate because project 7abad03e64cb47fba3d37d5f84b3c899 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:45.354614 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:45.355277 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:45.356908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1389/2797] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:45 2022] DELETE /identity/v3/projects/7abad03e64cb47fba3d37d5f84b3c899 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:45.361509 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e3015-cd74-4283-b983-8a6196f1560d service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:45.371904 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e3015-cd74-4283-b983-8a6196f1560d service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:45.429691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c7d00-0579-4b67-b4f9-33d95a074d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:45.430463 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c7d00-0579-4b67-b4f9-33d95a074d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:45.430739 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c7d00-0579-4b67-b4f9-33d95a074d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:45.431040 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c7d00-0579-4b67-b4f9-33d95a074d15 admin admin] PATH_INFO: `/v3/users/6042f3363bb64626ab3499582f47eb18` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:45.432162 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c7d00-0579-4b67-b4f9-33d95a074d15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6042f3363bb64626ab3499582f47eb18)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:45.434155 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c7d00-0579-4b67-b4f9-33d95a074d15 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:45.482648 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-bc9c7d00-0579-4b67-b4f9-33d95a074d15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6042f3363bb64626ab3499582f47eb18'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:45.491996 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-bc9c7d00-0579-4b67-b4f9-33d95a074d15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6042f3363bb64626ab3499582f47eb18'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:45.492820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-bc9c7d00-0579-4b67-b4f9-33d95a074d15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6042f3363bb64626ab3499582f47eb18'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:45.493521 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-bc9c7d00-0579-4b67-b4f9-33d95a074d15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6042f3363bb64626ab3499582f47eb18'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:45.497406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:45.498850 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:45.501507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-bc9c7d00-0579-4b67-b4f9-33d95a074d15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6042f3363bb64626ab3499582f47eb18'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:45.519241 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1409/2798] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:45 2022] DELETE /identity/v3/users/6042f3363bb64626ab3499582f47eb18 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:45.574000 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647a051-d9fc-4a3e-858e-ebcad5c06cfc admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:45.637390 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509f282-2327-4b36-892c-b90086233ba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:45.638005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509f282-2327-4b36-892c-b90086233ba6 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:45.638279 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509f282-2327-4b36-892c-b90086233ba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:45.638544 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509f282-2327-4b36-892c-b90086233ba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:45.640309 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3509f282-2327-4b36-892c-b90086233ba6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:45.675783 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:45.713531 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509f282-2327-4b36-892c-b90086233ba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:45.715077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509f282-2327-4b36-892c-b90086233ba6 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:45.725032 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1390/2799] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:45.841982 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c7d00-0579-4b67-b4f9-33d95a074d15 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:45.843512 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-bc9c7d00-0579-4b67-b4f9-33d95a074d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:45.920725 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c7d00-0579-4b67-b4f9-33d95a074d15 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:45.925249 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a79007-da4d-46a4-83b3-d8e31d42d2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:45.925880 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a79007-da4d-46a4-83b3-d8e31d42d2e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:45.926157 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a79007-da4d-46a4-83b3-d8e31d42d2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:45.926427 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a79007-da4d-46a4-83b3-d8e31d42d2e0 admin admin] PATH_INFO: `/v3/projects/d7e4def861fe47e2ba5b954524777182` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:45.927527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a79007-da4d-46a4-83b3-d8e31d42d2e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7e4def861fe47e2ba5b954524777182)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:45.929483 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a79007-da4d-46a4-83b3-d8e31d42d2e0 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:45.951700 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:45.980904 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-20a79007-da4d-46a4-83b3-d8e31d42d2e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7e4def861fe47e2ba5b954524777182', 'request_id': 'req-20a79007-da4d-46a4-83b3-d8e31d42d2e0'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:45.987059 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-20a79007-da4d-46a4-83b3-d8e31d42d2e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7e4def861fe47e2ba5b954524777182', 'request_id': 'req-20a79007-da4d-46a4-83b3-d8e31d42d2e0'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:45.995516 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-20a79007-da4d-46a4-83b3-d8e31d42d2e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7e4def861fe47e2ba5b954524777182', 'request_id': 'req-20a79007-da4d-46a4-83b3-d8e31d42d2e0'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:46.006137 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-20a79007-da4d-46a4-83b3-d8e31d42d2e0 admin admin] The token cache is being invalidate because project d7e4def861fe47e2ba5b954524777182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:46.006423 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-20a79007-da4d-46a4-83b3-d8e31d42d2e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:46.007037 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-20a79007-da4d-46a4-83b3-d8e31d42d2e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:46.008516 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1410/2800] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:45 2022] DELETE /identity/v3/projects/d7e4def861fe47e2ba5b954524777182 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:46.012761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509f282-2327-4b36-892c-b90086233ba6 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:46.025292 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509f282-2327-4b36-892c-b90086233ba6 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:46.082913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04e234-cf68-47bc-a90f-9e50ffc668aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:46.083557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04e234-cf68-47bc-a90f-9e50ffc668aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:46.083837 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04e234-cf68-47bc-a90f-9e50ffc668aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:46.084110 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04e234-cf68-47bc-a90f-9e50ffc668aa admin admin] PATH_INFO: `/v3/users/262b5b4ecd8f4a94a35b6714acc42d5a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:46.085331 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04e234-cf68-47bc-a90f-9e50ffc668aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=262b5b4ecd8f4a94a35b6714acc42d5a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:46.087270 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04e234-cf68-47bc-a90f-9e50ffc668aa admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:46.138021 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7a04e234-cf68-47bc-a90f-9e50ffc668aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '262b5b4ecd8f4a94a35b6714acc42d5a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:46.147081 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7a04e234-cf68-47bc-a90f-9e50ffc668aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '262b5b4ecd8f4a94a35b6714acc42d5a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:46.147904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7a04e234-cf68-47bc-a90f-9e50ffc668aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '262b5b4ecd8f4a94a35b6714acc42d5a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:46.148708 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7a04e234-cf68-47bc-a90f-9e50ffc668aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '262b5b4ecd8f4a94a35b6714acc42d5a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:46.166166 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7a04e234-cf68-47bc-a90f-9e50ffc668aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '262b5b4ecd8f4a94a35b6714acc42d5a'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:46.178955 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1391/2801] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:46 2022] DELETE /identity/v3/users/262b5b4ecd8f4a94a35b6714acc42d5a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:46.468311 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a79007-da4d-46a4-83b3-d8e31d42d2e0 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:46.470458 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-20a79007-da4d-46a4-83b3-d8e31d42d2e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:46.485251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a79007-da4d-46a4-83b3-d8e31d42d2e0 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:46.545923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff709bcb-e1c3-4287-80d8-f0ad260e579f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:46.546560 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff709bcb-e1c3-4287-80d8-f0ad260e579f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:46.546838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff709bcb-e1c3-4287-80d8-f0ad260e579f admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:46.547107 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff709bcb-e1c3-4287-80d8-f0ad260e579f admin admin] PATH_INFO: `/v3/projects/6b9cd3e7f77c48db874a151d9ab07a21` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:46.548639 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff709bcb-e1c3-4287-80d8-f0ad260e579f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b9cd3e7f77c48db874a151d9ab07a21)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:46.551415 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff709bcb-e1c3-4287-80d8-f0ad260e579f admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:46.604248 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-ff709bcb-e1c3-4287-80d8-f0ad260e579f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b9cd3e7f77c48db874a151d9ab07a21', 'request_id': 'req-ff709bcb-e1c3-4287-80d8-f0ad260e579f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:46.610826 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-ff709bcb-e1c3-4287-80d8-f0ad260e579f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b9cd3e7f77c48db874a151d9ab07a21', 'request_id': 'req-ff709bcb-e1c3-4287-80d8-f0ad260e579f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:46.621269 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-ff709bcb-e1c3-4287-80d8-f0ad260e579f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b9cd3e7f77c48db874a151d9ab07a21', 'request_id': 'req-ff709bcb-e1c3-4287-80d8-f0ad260e579f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:46.634006 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-ff709bcb-e1c3-4287-80d8-f0ad260e579f admin admin] The token cache is being invalidate because project 6b9cd3e7f77c48db874a151d9ab07a21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:46.634471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-ff709bcb-e1c3-4287-80d8-f0ad260e579f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:46.634920 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-ff709bcb-e1c3-4287-80d8-f0ad260e579f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:46.636438 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1411/2802] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:46 2022] DELETE /identity/v3/projects/6b9cd3e7f77c48db874a151d9ab07a21 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:46.642567 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429edcff-0256-4c43-9391-91bd08cc6b3f None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:46.643039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429edcff-0256-4c43-9391-91bd08cc6b3f None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:46.643127 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429edcff-0256-4c43-9391-91bd08cc6b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:46.664477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-429edcff-0256-4c43-9391-91bd08cc6b3f None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:46.665779 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-429edcff-0256-4c43-9391-91bd08cc6b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:46.716796 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:46.741395 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1392/2803] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:46.750457 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b22df-48cd-475d-a2e9-5fe90ad67fe2 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:46.750743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b22df-48cd-475d-a2e9-5fe90ad67fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:46.751017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b22df-48cd-475d-a2e9-5fe90ad67fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:46.775535 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-2b3b22df-48cd-475d-a2e9-5fe90ad67fe2 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:46.777469 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2b3b22df-48cd-475d-a2e9-5fe90ad67fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:46.787584 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1412/2804] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:46.792405 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429edcff-0256-4c43-9391-91bd08cc6b3f None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:46.803126 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429edcff-0256-4c43-9391-91bd08cc6b3f None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:46.809594 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5248299-0a20-47e6-b04f-8e23f1d30d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:46.810203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5248299-0a20-47e6-b04f-8e23f1d30d78 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:46.810478 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5248299-0a20-47e6-b04f-8e23f1d30d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:46.810752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5248299-0a20-47e6-b04f-8e23f1d30d78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:46.811398 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5248299-0a20-47e6-b04f-8e23f1d30d78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:56:46.811707 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5248299-0a20-47e6-b04f-8e23f1d30d78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:46.813425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5248299-0a20-47e6-b04f-8e23f1d30d78 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:46.826285 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1393/2805] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:56:46.831324 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b22df-48cd-475d-a2e9-5fe90ad67fe2 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:46.840762 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b22df-48cd-475d-a2e9-5fe90ad67fe2 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:46.846705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809d3ec-f0d9-49c6-96f0-cbcb74ed15a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:46.847312 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809d3ec-f0d9-49c6-96f0-cbcb74ed15a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:46.847569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809d3ec-f0d9-49c6-96f0-cbcb74ed15a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:46.847866 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809d3ec-f0d9-49c6-96f0-cbcb74ed15a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:46.848478 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809d3ec-f0d9-49c6-96f0-cbcb74ed15a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:46.850435 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809d3ec-f0d9-49c6-96f0-cbcb74ed15a0 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:46.883276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1413/2806] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:46 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:46.887642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5248299-0a20-47e6-b04f-8e23f1d30d78 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:46.898910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5248299-0a20-47e6-b04f-8e23f1d30d78 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:46.964636 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d249b93-007c-46ca-8235-0878a854e5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:46.965447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d249b93-007c-46ca-8235-0878a854e5cd admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:46.965802 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d249b93-007c-46ca-8235-0878a854e5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:46.966083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d249b93-007c-46ca-8235-0878a854e5cd admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:46.966851 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d249b93-007c-46ca-8235-0878a854e5cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:46.969420 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d249b93-007c-46ca-8235-0878a854e5cd admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:46.992769 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1394/2807] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:46 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:46.997246 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809d3ec-f0d9-49c6-96f0-cbcb74ed15a0 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:47.008199 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809d3ec-f0d9-49c6-96f0-cbcb74ed15a0 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:47.012981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a16d7-e8f4-4928-bf32-43b1d71933cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:47.013907 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a16d7-e8f4-4928-bf32-43b1d71933cf admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:47.014089 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a16d7-e8f4-4928-bf32-43b1d71933cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:47.014468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a16d7-e8f4-4928-bf32-43b1d71933cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:47.015517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a16d7-e8f4-4928-bf32-43b1d71933cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:47.017706 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a16d7-e8f4-4928-bf32-43b1d71933cf admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:47.020701 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:47.034696 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1414/2808] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:47.035704 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d249b93-007c-46ca-8235-0878a854e5cd admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:47.046527 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d249b93-007c-46ca-8235-0878a854e5cd admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:47.052046 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1f81c-2b3a-4f8f-9962-104328b88999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:47.052727 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1f81c-2b3a-4f8f-9962-104328b88999 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:47.053077 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1f81c-2b3a-4f8f-9962-104328b88999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:47.053518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1f81c-2b3a-4f8f-9962-104328b88999 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:47.054233 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1f81c-2b3a-4f8f-9962-104328b88999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:47.055970 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1f81c-2b3a-4f8f-9962-104328b88999 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:47.067608 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1395/2809] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:47.072512 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a16d7-e8f4-4928-bf32-43b1d71933cf admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:47.084025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a16d7-e8f4-4928-bf32-43b1d71933cf admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:47.091904 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35380cde-f34c-41ec-88a4-7afc6f1b81b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:47.092965 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35380cde-f34c-41ec-88a4-7afc6f1b81b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:47.093486 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35380cde-f34c-41ec-88a4-7afc6f1b81b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:47.093777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35380cde-f34c-41ec-88a4-7afc6f1b81b5 admin admin] PATH_INFO: `/v3/projects/acebd9727c4d4e83b711ddcf6a3d69d2/users/724bcfaa7af8470db1aa35378acd52c7/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:47.130788 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35380cde-f34c-41ec-88a4-7afc6f1b81b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acebd9727c4d4e83b711ddcf6a3d69d2, user_id=724bcfaa7af8470db1aa35378acd52c7, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:47.136662 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35380cde-f34c-41ec-88a4-7afc6f1b81b5 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:47.156876 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1415/2810] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:47 2022] PUT /identity/v3/projects/acebd9727c4d4e83b711ddcf6a3d69d2/users/724bcfaa7af8470db1aa35378acd52c7/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:47.170483 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011deca-8fdb-4273-95aa-3d1bfb0919cb None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:47.170881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011deca-8fdb-4273-95aa-3d1bfb0919cb None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:47.171189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011deca-8fdb-4273-95aa-3d1bfb0919cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:47.223573 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-5011deca-8fdb-4273-95aa-3d1bfb0919cb None None] MFA Rules not processed for user `724bcfaa7af8470db1aa35378acd52c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:47.225006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5011deca-8fdb-4273-95aa-3d1bfb0919cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:47.289338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1396/2811] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:47.294781 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35380cde-f34c-41ec-88a4-7afc6f1b81b5 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:47.357662 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35380cde-f34c-41ec-88a4-7afc6f1b81b5 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:47.363331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216a7a4-6218-4f02-8f87-e6d9c22c5bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:47.363928 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216a7a4-6218-4f02-8f87-e6d9c22c5bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:47.364205 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216a7a4-6218-4f02-8f87-e6d9c22c5bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:47.364487 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216a7a4-6218-4f02-8f87-e6d9c22c5bf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:47.365105 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b216a7a4-6218-4f02-8f87-e6d9c22c5bf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:47.367017 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b216a7a4-6218-4f02-8f87-e6d9c22c5bf8 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:47.402127 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1416/2812] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:47 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:47.408311 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011deca-8fdb-4273-95aa-3d1bfb0919cb None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:47.424513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011deca-8fdb-4273-95aa-3d1bfb0919cb None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:47.484878 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:47.502338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16428e-64cb-4e51-a335-81e9a3f4f463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:47.503193 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16428e-64cb-4e51-a335-81e9a3f4f463 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:47.503634 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16428e-64cb-4e51-a335-81e9a3f4f463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:47.504034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16428e-64cb-4e51-a335-81e9a3f4f463 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:47.504874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16428e-64cb-4e51-a335-81e9a3f4f463 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:47.506872 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16428e-64cb-4e51-a335-81e9a3f4f463 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:47.529661 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1397/2813] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:47 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:47.534400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216a7a4-6218-4f02-8f87-e6d9c22c5bf8 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:47.549553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216a7a4-6218-4f02-8f87-e6d9c22c5bf8 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:47.554454 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be00541-9e2b-44cd-89aa-609ea84851f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:47.555085 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be00541-9e2b-44cd-89aa-609ea84851f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:47.555373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be00541-9e2b-44cd-89aa-609ea84851f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:47.555649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be00541-9e2b-44cd-89aa-609ea84851f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:47.556289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be00541-9e2b-44cd-89aa-609ea84851f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:47.558027 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be00541-9e2b-44cd-89aa-609ea84851f9 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:47.569083 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1417/2814] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:47.573319 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16428e-64cb-4e51-a335-81e9a3f4f463 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:47.584240 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16428e-64cb-4e51-a335-81e9a3f4f463 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:47.589489 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82df105-d601-4e3f-9504-d7ad50f525ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:47.590116 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82df105-d601-4e3f-9504-d7ad50f525ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:47.590396 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82df105-d601-4e3f-9504-d7ad50f525ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:47.590641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82df105-d601-4e3f-9504-d7ad50f525ef admin admin] PATH_INFO: `/v3/projects/7392c61336234f2daf6e93bd5016767c/users/cad5fe587fb7488693656477f4379b8b/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:47.606370 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82df105-d601-4e3f-9504-d7ad50f525ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=7392c61336234f2daf6e93bd5016767c, user_id=cad5fe587fb7488693656477f4379b8b, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:47.609021 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82df105-d601-4e3f-9504-d7ad50f525ef admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:47.624163 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1398/2815] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:47 2022] PUT /identity/v3/projects/7392c61336234f2daf6e93bd5016767c/users/cad5fe587fb7488693656477f4379b8b/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:47.628599 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be00541-9e2b-44cd-89aa-609ea84851f9 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:47.639922 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be00541-9e2b-44cd-89aa-609ea84851f9 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:47.712777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a17f87-f5e4-4349-93ce-6366806246af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:47.713427 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a17f87-f5e4-4349-93ce-6366806246af admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:47.713705 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a17f87-f5e4-4349-93ce-6366806246af admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:47.713971 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a17f87-f5e4-4349-93ce-6366806246af admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:47.714611 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a17f87-f5e4-4349-93ce-6366806246af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:47.716176 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a17f87-f5e4-4349-93ce-6366806246af admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:47.730647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1418/2816] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:47.737061 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82df105-d601-4e3f-9504-d7ad50f525ef admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:47.751937 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82df105-d601-4e3f-9504-d7ad50f525ef admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:47.758021 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc77595-32ce-4647-9b39-571a743803e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:47.758709 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc77595-32ce-4647-9b39-571a743803e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:47.758980 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc77595-32ce-4647-9b39-571a743803e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:47.759251 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc77595-32ce-4647-9b39-571a743803e2 admin admin] PATH_INFO: `/v3/projects/7392c61336234f2daf6e93bd5016767c/users/cad5fe587fb7488693656477f4379b8b/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:47.761894 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc77595-32ce-4647-9b39-571a743803e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7392c61336234f2daf6e93bd5016767c, user_id=cad5fe587fb7488693656477f4379b8b, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:47.764547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc77595-32ce-4647-9b39-571a743803e2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:47.775934 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1399/2817] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:47 2022] PUT /identity/v3/projects/7392c61336234f2daf6e93bd5016767c/users/cad5fe587fb7488693656477f4379b8b/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:47.858845 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08870c-78d1-43bb-ae53-56a4de7efa42 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:47.859777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08870c-78d1-43bb-ae53-56a4de7efa42 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:47.859777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08870c-78d1-43bb-ae53-56a4de7efa42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:47.896021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-9b08870c-78d1-43bb-ae53-56a4de7efa42 None None] MFA Rules not processed for user `cad5fe587fb7488693656477f4379b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:47.897271 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9b08870c-78d1-43bb-ae53-56a4de7efa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:47.931735 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:47.981803 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1419/2818] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:47.993655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc77595-32ce-4647-9b39-571a743803e2 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:47.995948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8cc77595-32ce-4647-9b39-571a743803e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:48.055956 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc77595-32ce-4647-9b39-571a743803e2 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:48.059833 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d71084-e5bd-411e-8ad2-33875ff740cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:48.060434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d71084-e5bd-411e-8ad2-33875ff740cd service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:48.060719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d71084-e5bd-411e-8ad2-33875ff740cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:48.060990 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d71084-e5bd-411e-8ad2-33875ff740cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:48.121253 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d71084-e5bd-411e-8ad2-33875ff740cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:48.122750 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d71084-e5bd-411e-8ad2-33875ff740cd service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:48.131927 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1400/2819] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:49.743693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08870c-78d1-43bb-ae53-56a4de7efa42 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:49.745365 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9b08870c-78d1-43bb-ae53-56a4de7efa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:49.822948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08870c-78d1-43bb-ae53-56a4de7efa42 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:49.828383 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5d054-84e2-4363-bb26-79da35900818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:49.829367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5d054-84e2-4363-bb26-79da35900818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:49.829660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5d054-84e2-4363-bb26-79da35900818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:49.829927 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5d054-84e2-4363-bb26-79da35900818 admin admin] PATH_INFO: `/v3/users/c9fa90d0142145b79eb6ee5b3926b9a9` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:49.831108 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f5d054-84e2-4363-bb26-79da35900818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9fa90d0142145b79eb6ee5b3926b9a9)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:49.833020 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f5d054-84e2-4363-bb26-79da35900818 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:49.891232 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f3f5d054-84e2-4363-bb26-79da35900818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9fa90d0142145b79eb6ee5b3926b9a9'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:49.899605 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f3f5d054-84e2-4363-bb26-79da35900818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9fa90d0142145b79eb6ee5b3926b9a9'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:49.900263 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f3f5d054-84e2-4363-bb26-79da35900818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9fa90d0142145b79eb6ee5b3926b9a9'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:49.900904 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f3f5d054-84e2-4363-bb26-79da35900818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9fa90d0142145b79eb6ee5b3926b9a9'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:49.908667 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-f3f5d054-84e2-4363-bb26-79da35900818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9fa90d0142145b79eb6ee5b3926b9a9'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:49.926411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1420/2820] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:49 2022] DELETE /identity/v3/users/c9fa90d0142145b79eb6ee5b3926b9a9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:49.931615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ffe50-e1d7-4ff1-ba26-7521828f9622 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:49.932147 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ffe50-e1d7-4ff1-ba26-7521828f9622 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:49.932251 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ffe50-e1d7-4ff1-ba26-7521828f9622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:49.952958 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-172ffe50-e1d7-4ff1-ba26-7521828f9622 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:49.954719 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-172ffe50-e1d7-4ff1-ba26-7521828f9622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:50.006434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:50.020383 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1401/2821] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:50.030223 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5d054-84e2-4363-bb26-79da35900818 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:50.031872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f3f5d054-84e2-4363-bb26-79da35900818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:50.087537 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:50.092948 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5d054-84e2-4363-bb26-79da35900818 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:50.098891 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0785860-6e14-4c92-918e-2fc140627e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:50.099696 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0785860-6e14-4c92-918e-2fc140627e8d service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:50.100124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0785860-6e14-4c92-918e-2fc140627e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:50.100391 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0785860-6e14-4c92-918e-2fc140627e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:50.118282 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0785860-6e14-4c92-918e-2fc140627e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:50.120305 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0785860-6e14-4c92-918e-2fc140627e8d service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:50.133175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1421/2822] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:50.422943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ffe50-e1d7-4ff1-ba26-7521828f9622 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:50.424597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-172ffe50-e1d7-4ff1-ba26-7521828f9622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:50.438768 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ffe50-e1d7-4ff1-ba26-7521828f9622 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:50.444493 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:50.445197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:50.445479 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:50.445745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d admin admin] PATH_INFO: `/v3/projects/abb459a3fade4a088724d66d43dd1067` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:50.447028 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb459a3fade4a088724d66d43dd1067)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:50.448950 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:50.496777 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb459a3fade4a088724d66d43dd1067', 'request_id': 'req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:50.503471 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb459a3fade4a088724d66d43dd1067', 'request_id': 'req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:50.513978 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb459a3fade4a088724d66d43dd1067', 'request_id': 'req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:50.532501 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d admin admin] The token cache is being invalidate because project abb459a3fade4a088724d66d43dd1067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:50.532752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:50.533493 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:50.535124 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1402/2823] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:50 2022] DELETE /identity/v3/projects/abb459a3fade4a088724d66d43dd1067 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:50.539296 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0785860-6e14-4c92-918e-2fc140627e8d service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:50.549693 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0785860-6e14-4c92-918e-2fc140627e8d service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:50.615258 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba7b50-6dd9-4851-a640-2045fd7e9222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:50.616175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba7b50-6dd9-4851-a640-2045fd7e9222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:50.616545 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba7b50-6dd9-4851-a640-2045fd7e9222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:50.616915 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba7b50-6dd9-4851-a640-2045fd7e9222 admin admin] PATH_INFO: `/v3/users/b278dc1130054323a250ab8af3042fd3` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:50.618840 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba7b50-6dd9-4851-a640-2045fd7e9222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b278dc1130054323a250ab8af3042fd3)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:50.621890 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba7b50-6dd9-4851-a640-2045fd7e9222 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:50.681740 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0aba7b50-6dd9-4851-a640-2045fd7e9222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b278dc1130054323a250ab8af3042fd3'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:50.690523 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0aba7b50-6dd9-4851-a640-2045fd7e9222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b278dc1130054323a250ab8af3042fd3'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:50.691167 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0aba7b50-6dd9-4851-a640-2045fd7e9222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b278dc1130054323a250ab8af3042fd3'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:50.691804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0aba7b50-6dd9-4851-a640-2045fd7e9222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b278dc1130054323a250ab8af3042fd3'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:50.698862 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-0aba7b50-6dd9-4851-a640-2045fd7e9222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b278dc1130054323a250ab8af3042fd3'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:50.715944 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1422/2824] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:50 2022] DELETE /identity/v3/users/b278dc1130054323a250ab8af3042fd3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:50.757761 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:50.759228 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:50.773099 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d766e-7f05-4ea8-9d2f-826000cfa02d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:50.839278 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a02cf-6192-4270-8ceb-d71f9a391391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:50.839982 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a02cf-6192-4270-8ceb-d71f9a391391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:50.840268 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a02cf-6192-4270-8ceb-d71f9a391391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:50.840538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a02cf-6192-4270-8ceb-d71f9a391391 admin admin] PATH_INFO: `/v3/users/724bcfaa7af8470db1aa35378acd52c7` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:50.841726 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a02cf-6192-4270-8ceb-d71f9a391391 admin admin] RBAC: Authorizing `identity:delete_user(user_id=724bcfaa7af8470db1aa35378acd52c7)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:50.843708 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a02cf-6192-4270-8ceb-d71f9a391391 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:50.937533 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ba0a02cf-6192-4270-8ceb-d71f9a391391 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '724bcfaa7af8470db1aa35378acd52c7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:50.946597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ba0a02cf-6192-4270-8ceb-d71f9a391391 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '724bcfaa7af8470db1aa35378acd52c7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:50.947258 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ba0a02cf-6192-4270-8ceb-d71f9a391391 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '724bcfaa7af8470db1aa35378acd52c7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:50.948071 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ba0a02cf-6192-4270-8ceb-d71f9a391391 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '724bcfaa7af8470db1aa35378acd52c7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:50.964175 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-ba0a02cf-6192-4270-8ceb-d71f9a391391 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '724bcfaa7af8470db1aa35378acd52c7'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:50.970240 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:50.979834 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1403/2825] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:50 2022] DELETE /identity/v3/users/724bcfaa7af8470db1aa35378acd52c7 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:50.984519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837d897-efcc-4b34-9213-aa6a7af359d9 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:50.985047 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837d897-efcc-4b34-9213-aa6a7af359d9 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:50.985349 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837d897-efcc-4b34-9213-aa6a7af359d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:51.005056 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-c837d897-efcc-4b34-9213-aa6a7af359d9 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:51.006287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c837d897-efcc-4b34-9213-aa6a7af359d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:51.038381 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a02cf-6192-4270-8ceb-d71f9a391391 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:51.039948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ba0a02cf-6192-4270-8ceb-d71f9a391391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:51.076449 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1423/2826] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:51.087079 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837d897-efcc-4b34-9213-aa6a7af359d9 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:51.088643 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c837d897-efcc-4b34-9213-aa6a7af359d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:51.104167 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:51.115364 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a02cf-6192-4270-8ceb-d71f9a391391 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:51.120332 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2b301-7f09-4f42-b977-7eeae12f8592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:51.121608 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2b301-7f09-4f42-b977-7eeae12f8592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:51.121608 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2b301-7f09-4f42-b977-7eeae12f8592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:51.121608 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2b301-7f09-4f42-b977-7eeae12f8592 admin admin] PATH_INFO: `/v3/projects/cc464062efa544128155fe6cb226ba3b` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:51.122663 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2b301-7f09-4f42-b977-7eeae12f8592 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc464062efa544128155fe6cb226ba3b)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:51.124565 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2b301-7f09-4f42-b977-7eeae12f8592 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:51.162318 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837d897-efcc-4b34-9213-aa6a7af359d9 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:51.166956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14017e56-7854-401b-89b1-178904868a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:51.168030 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14017e56-7854-401b-89b1-178904868a44 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:51.168571 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14017e56-7854-401b-89b1-178904868a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:51.168903 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14017e56-7854-401b-89b1-178904868a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:51.179292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14017e56-7854-401b-89b1-178904868a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:51.180720 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14017e56-7854-401b-89b1-178904868a44 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:51.188617 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fef2b301-7f09-4f42-b977-7eeae12f8592 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc464062efa544128155fe6cb226ba3b', 'request_id': 'req-fef2b301-7f09-4f42-b977-7eeae12f8592'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.190289 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1424/2827] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:56:51.195196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fef2b301-7f09-4f42-b977-7eeae12f8592 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc464062efa544128155fe6cb226ba3b', 'request_id': 'req-fef2b301-7f09-4f42-b977-7eeae12f8592'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.204640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fef2b301-7f09-4f42-b977-7eeae12f8592 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc464062efa544128155fe6cb226ba3b', 'request_id': 'req-fef2b301-7f09-4f42-b977-7eeae12f8592'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.216220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fef2b301-7f09-4f42-b977-7eeae12f8592 admin admin] The token cache is being invalidate because project cc464062efa544128155fe6cb226ba3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:51.216513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fef2b301-7f09-4f42-b977-7eeae12f8592 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.217341 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fef2b301-7f09-4f42-b977-7eeae12f8592 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.218866 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1404/2828] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:51 2022] DELETE /identity/v3/projects/cc464062efa544128155fe6cb226ba3b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:51.222977 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14017e56-7854-401b-89b1-178904868a44 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:51.233972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14017e56-7854-401b-89b1-178904868a44 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:51.294083 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf616b-7fcb-49e3-b475-32aaa209268d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:51.294695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf616b-7fcb-49e3-b475-32aaa209268d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:51.294982 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf616b-7fcb-49e3-b475-32aaa209268d admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:51.295209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf616b-7fcb-49e3-b475-32aaa209268d admin admin] PATH_INFO: `/v3/users/ee002959335f4cb788b9a3d5a7f99ba8` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:51.296400 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf616b-7fcb-49e3-b475-32aaa209268d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee002959335f4cb788b9a3d5a7f99ba8)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:51.298378 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf616b-7fcb-49e3-b475-32aaa209268d admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:51.347527 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-bfbf616b-7fcb-49e3-b475-32aaa209268d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee002959335f4cb788b9a3d5a7f99ba8'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.355871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-bfbf616b-7fcb-49e3-b475-32aaa209268d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee002959335f4cb788b9a3d5a7f99ba8'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.356552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-bfbf616b-7fcb-49e3-b475-32aaa209268d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee002959335f4cb788b9a3d5a7f99ba8'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.357187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-bfbf616b-7fcb-49e3-b475-32aaa209268d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee002959335f4cb788b9a3d5a7f99ba8'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.364710 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-bfbf616b-7fcb-49e3-b475-32aaa209268d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee002959335f4cb788b9a3d5a7f99ba8'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.412534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1425/2829] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:51 2022] DELETE /identity/v3/users/ee002959335f4cb788b9a3d5a7f99ba8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:51.514915 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2b301-7f09-4f42-b977-7eeae12f8592 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:51.516908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-fef2b301-7f09-4f42-b977-7eeae12f8592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:51.542326 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2b301-7f09-4f42-b977-7eeae12f8592 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:51.606791 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:51.607447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:51.607724 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:51.608164 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d admin admin] PATH_INFO: `/v3/projects/acebd9727c4d4e83b711ddcf6a3d69d2` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:51.609355 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=acebd9727c4d4e83b711ddcf6a3d69d2)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:51.611413 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:51.656156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:51.680171 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acebd9727c4d4e83b711ddcf6a3d69d2', 'request_id': 'req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.687184 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acebd9727c4d4e83b711ddcf6a3d69d2', 'request_id': 'req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.697298 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acebd9727c4d4e83b711ddcf6a3d69d2', 'request_id': 'req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.715418 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d admin admin] The token cache is being invalidate because project acebd9727c4d4e83b711ddcf6a3d69d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:51.715812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.716462 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.718064 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1405/2830] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:51 2022] DELETE /identity/v3/projects/acebd9727c4d4e83b711ddcf6a3d69d2 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:51.722322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf616b-7fcb-49e3-b475-32aaa209268d admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:51.723748 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-bfbf616b-7fcb-49e3-b475-32aaa209268d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:51.727745 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:51.729465 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:51.737717 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf616b-7fcb-49e3-b475-32aaa209268d admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:51.778043 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:51.802366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3ddc1-8bdb-4d91-85ed-d2e169776a8d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:51.805148 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc04ae-93fb-457d-925e-a756c9138b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:51.805799 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc04ae-93fb-457d-925e-a756c9138b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:51.806072 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc04ae-93fb-457d-925e-a756c9138b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:51.806338 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc04ae-93fb-457d-925e-a756c9138b3e admin admin] PATH_INFO: `/v3/users/cad5fe587fb7488693656477f4379b8b` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:51.807179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128fc33-0c86-4af2-8bd5-a4b36ff1253e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:51.807471 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc04ae-93fb-457d-925e-a756c9138b3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cad5fe587fb7488693656477f4379b8b)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:51.807889 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128fc33-0c86-4af2-8bd5-a4b36ff1253e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:51.808169 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128fc33-0c86-4af2-8bd5-a4b36ff1253e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:51.808468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128fc33-0c86-4af2-8bd5-a4b36ff1253e admin admin] PATH_INFO: `/v3/projects/fd76c42ed5da43b58cce0feddbfee27e` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:51.809515 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc04ae-93fb-457d-925e-a756c9138b3e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:51.809625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128fc33-0c86-4af2-8bd5-a4b36ff1253e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd76c42ed5da43b58cce0feddbfee27e)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:51.811538 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128fc33-0c86-4af2-8bd5-a4b36ff1253e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:51.857979 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-87dc04ae-93fb-457d-925e-a756c9138b3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cad5fe587fb7488693656477f4379b8b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.861765 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9128fc33-0c86-4af2-8bd5-a4b36ff1253e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd76c42ed5da43b58cce0feddbfee27e', 'request_id': 'req-9128fc33-0c86-4af2-8bd5-a4b36ff1253e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.865972 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-87dc04ae-93fb-457d-925e-a756c9138b3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cad5fe587fb7488693656477f4379b8b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.866880 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-87dc04ae-93fb-457d-925e-a756c9138b3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cad5fe587fb7488693656477f4379b8b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.867683 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-87dc04ae-93fb-457d-925e-a756c9138b3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cad5fe587fb7488693656477f4379b8b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.868276 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9128fc33-0c86-4af2-8bd5-a4b36ff1253e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd76c42ed5da43b58cce0feddbfee27e', 'request_id': 'req-9128fc33-0c86-4af2-8bd5-a4b36ff1253e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.875604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-87dc04ae-93fb-457d-925e-a756c9138b3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cad5fe587fb7488693656477f4379b8b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.877674 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9128fc33-0c86-4af2-8bd5-a4b36ff1253e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd76c42ed5da43b58cce0feddbfee27e', 'request_id': 'req-9128fc33-0c86-4af2-8bd5-a4b36ff1253e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.893843 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9128fc33-0c86-4af2-8bd5-a4b36ff1253e admin admin] The token cache is being invalidate because project fd76c42ed5da43b58cce0feddbfee27e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:51.894236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9128fc33-0c86-4af2-8bd5-a4b36ff1253e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.895106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-9128fc33-0c86-4af2-8bd5-a4b36ff1253e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:51.896669 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1406/2831] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:51 2022] DELETE /identity/v3/projects/fd76c42ed5da43b58cce0feddbfee27e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:51.902449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565ea0f-6492-4c2b-b40b-3dac89a30ab1 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:51.902891 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565ea0f-6492-4c2b-b40b-3dac89a30ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:51.903173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565ea0f-6492-4c2b-b40b-3dac89a30ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:51.905121 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1426/2832] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:56:51 2022] DELETE /identity/v3/users/cad5fe587fb7488693656477f4379b8b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:51.926669 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-5565ea0f-6492-4c2b-b40b-3dac89a30ab1 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:51.927920 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-5565ea0f-6492-4c2b-b40b-3dac89a30ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:51.978758 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:51.999296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1407/2833] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:52.007844 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293a801-0592-4b65-b7a1-2bf8aa94fc7b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.008380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293a801-0592-4b65-b7a1-2bf8aa94fc7b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.008380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293a801-0592-4b65-b7a1-2bf8aa94fc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.028839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-f293a801-0592-4b65-b7a1-2bf8aa94fc7b None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:52.030140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f293a801-0592-4b65-b7a1-2bf8aa94fc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:52.038026 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1427/2834] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:52.042725 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565ea0f-6492-4c2b-b40b-3dac89a30ab1 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:52.052601 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565ea0f-6492-4c2b-b40b-3dac89a30ab1 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:52.057066 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c55b3-0e48-48fc-9bb4-43a604880f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:52.057718 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c55b3-0e48-48fc-9bb4-43a604880f05 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.058448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c55b3-0e48-48fc-9bb4-43a604880f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.058448 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c55b3-0e48-48fc-9bb4-43a604880f05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.058913 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c55b3-0e48-48fc-9bb4-43a604880f05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:56:52.059220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c55b3-0e48-48fc-9bb4-43a604880f05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:52.060892 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c55b3-0e48-48fc-9bb4-43a604880f05 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:52.073805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1408/2835] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:56:52.078151 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293a801-0592-4b65-b7a1-2bf8aa94fc7b None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:52.088446 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293a801-0592-4b65-b7a1-2bf8aa94fc7b None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:52.093660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac67db-b9a1-4ac8-9853-28e0e1e6e307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:52.094277 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac67db-b9a1-4ac8-9853-28e0e1e6e307 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.094553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac67db-b9a1-4ac8-9853-28e0e1e6e307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.094818 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac67db-b9a1-4ac8-9853-28e0e1e6e307 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.095490 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac67db-b9a1-4ac8-9853-28e0e1e6e307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:52.097445 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac67db-b9a1-4ac8-9853-28e0e1e6e307 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:52.128000 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1428/2836] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:52 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 05:56:52.135194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c55b3-0e48-48fc-9bb4-43a604880f05 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:52.153083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c55b3-0e48-48fc-9bb4-43a604880f05 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:52.219519 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac67db-b9a1-4ac8-9853-28e0e1e6e307 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:52.221475 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-62ac67db-b9a1-4ac8-9853-28e0e1e6e307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:52.227421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300926b-cffd-4d1a-9e65-fa623949b11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:52.228080 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300926b-cffd-4d1a-9e65-fa623949b11b admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.228362 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300926b-cffd-4d1a-9e65-fa623949b11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.228636 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300926b-cffd-4d1a-9e65-fa623949b11b admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.231574 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300926b-cffd-4d1a-9e65-fa623949b11b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:52.233434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300926b-cffd-4d1a-9e65-fa623949b11b admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:52.259494 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1409/2837] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:52 2022] POST /identity/v3/users => generated 283 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:52.263600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300926b-cffd-4d1a-9e65-fa623949b11b admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:52.273425 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300926b-cffd-4d1a-9e65-fa623949b11b admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:52.278915 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4142b3-5909-4e52-9a06-1952044a2667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:52.279520 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4142b3-5909-4e52-9a06-1952044a2667 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.280175 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4142b3-5909-4e52-9a06-1952044a2667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.280431 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4142b3-5909-4e52-9a06-1952044a2667 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.281076 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4142b3-5909-4e52-9a06-1952044a2667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:52.282715 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4142b3-5909-4e52-9a06-1952044a2667 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:52.294027 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1410/2838] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:52.297616 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4142b3-5909-4e52-9a06-1952044a2667 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:52.306872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac67db-b9a1-4ac8-9853-28e0e1e6e307 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:52.307139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4142b3-5909-4e52-9a06-1952044a2667 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:52.311660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72731c47-53be-4134-a302-8e07fc4a8a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:52.312175 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595555f-6c62-4fd4-ab19-2cbbc6c1a428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:52.312308 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72731c47-53be-4134-a302-8e07fc4a8a2e admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.312506 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72731c47-53be-4134-a302-8e07fc4a8a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.312770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72731c47-53be-4134-a302-8e07fc4a8a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.312869 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595555f-6c62-4fd4-ab19-2cbbc6c1a428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.313069 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595555f-6c62-4fd4-ab19-2cbbc6c1a428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.313790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595555f-6c62-4fd4-ab19-2cbbc6c1a428 admin admin] PATH_INFO: `/v3/projects/7392c61336234f2daf6e93bd5016767c` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.313926 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72731c47-53be-4134-a302-8e07fc4a8a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:52.314901 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595555f-6c62-4fd4-ab19-2cbbc6c1a428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7392c61336234f2daf6e93bd5016767c)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:52.315518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72731c47-53be-4134-a302-8e07fc4a8a2e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:52.316897 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595555f-6c62-4fd4-ab19-2cbbc6c1a428 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:52.329530 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1411/2839] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:52.330560 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72731c47-53be-4134-a302-8e07fc4a8a2e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:52.340311 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72731c47-53be-4134-a302-8e07fc4a8a2e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:52.345989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a49213-562e-4207-832c-36abddfc21a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:52.346780 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a49213-562e-4207-832c-36abddfc21a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.347194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a49213-562e-4207-832c-36abddfc21a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.347473 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a49213-562e-4207-832c-36abddfc21a6 admin admin] PATH_INFO: `/v3/projects/ca0f07bf39e94f60b1aa8524098c89f9/users/2bf60c63ad54497dba7a1b8dbdc5f6a1/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.349979 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:52.365394 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a49213-562e-4207-832c-36abddfc21a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca0f07bf39e94f60b1aa8524098c89f9, user_id=2bf60c63ad54497dba7a1b8dbdc5f6a1, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:52.368173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a49213-562e-4207-832c-36abddfc21a6 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:52.368357 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7595555f-6c62-4fd4-ab19-2cbbc6c1a428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7392c61336234f2daf6e93bd5016767c', 'request_id': 'req-7595555f-6c62-4fd4-ab19-2cbbc6c1a428'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:52.375524 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7595555f-6c62-4fd4-ab19-2cbbc6c1a428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7392c61336234f2daf6e93bd5016767c', 'request_id': 'req-7595555f-6c62-4fd4-ab19-2cbbc6c1a428'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:52.385886 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7595555f-6c62-4fd4-ab19-2cbbc6c1a428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7392c61336234f2daf6e93bd5016767c', 'request_id': 'req-7595555f-6c62-4fd4-ab19-2cbbc6c1a428'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:52.387194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1412/2840] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:52 2022] PUT /identity/v3/projects/ca0f07bf39e94f60b1aa8524098c89f9/users/2bf60c63ad54497dba7a1b8dbdc5f6a1/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:52.409337 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7595555f-6c62-4fd4-ab19-2cbbc6c1a428 admin admin] The token cache is being invalidate because project 7392c61336234f2daf6e93bd5016767c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:56:52.409631 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7595555f-6c62-4fd4-ab19-2cbbc6c1a428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:52.410328 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-7595555f-6c62-4fd4-ab19-2cbbc6c1a428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:56:52.411936 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1429/2841] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:56:52 2022] DELETE /identity/v3/projects/7392c61336234f2daf6e93bd5016767c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:56:52.417817 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67213d0e-0dac-420a-8c99-740fb062a4ac None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.418185 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67213d0e-0dac-420a-8c99-740fb062a4ac None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.418508 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67213d0e-0dac-420a-8c99-740fb062a4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.444090 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-67213d0e-0dac-420a-8c99-740fb062a4ac None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:52.445441 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-67213d0e-0dac-420a-8c99-740fb062a4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:52.462529 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd28891-298b-44bc-8ca6-a437febdeda0 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.462791 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd28891-298b-44bc-8ca6-a437febdeda0 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.463061 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd28891-298b-44bc-8ca6-a437febdeda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.492619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-9fd28891-298b-44bc-8ca6-a437febdeda0 None None] MFA Rules not processed for user `2bf60c63ad54497dba7a1b8dbdc5f6a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:52.493934 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9fd28891-298b-44bc-8ca6-a437febdeda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:52.511238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1413/2842] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:52.518896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130b8e1-c46d-4422-bf9b-6ccb7470534c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.519181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130b8e1-c46d-4422-bf9b-6ccb7470534c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.519453 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130b8e1-c46d-4422-bf9b-6ccb7470534c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.539256 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-c130b8e1-c46d-4422-bf9b-6ccb7470534c None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:52.540464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-c130b8e1-c46d-4422-bf9b-6ccb7470534c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:52.549088 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1414/2843] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:52.552948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130b8e1-c46d-4422-bf9b-6ccb7470534c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:52.562214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130b8e1-c46d-4422-bf9b-6ccb7470534c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:52.565881 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1430/2844] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:56:52.570214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1fbcc-9f60-4bab-9f8f-eb8ab47471f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:52.570214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1fbcc-9f60-4bab-9f8f-eb8ab47471f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.570214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1fbcc-9f60-4bab-9f8f-eb8ab47471f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.570214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1fbcc-9f60-4bab-9f8f-eb8ab47471f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.570639 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a1fbcc-9f60-4bab-9f8f-eb8ab47471f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:56:52.570639 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a1fbcc-9f60-4bab-9f8f-eb8ab47471f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:52.570819 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a1fbcc-9f60-4bab-9f8f-eb8ab47471f5 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:52.578639 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:52.578847 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd28891-298b-44bc-8ca6-a437febdeda0 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:52.582090 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9fd28891-298b-44bc-8ca6-a437febdeda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:52.589333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1415/2845] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 05:56:52.593623 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1fbcc-9f60-4bab-9f8f-eb8ab47471f5 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:52.604117 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1fbcc-9f60-4bab-9f8f-eb8ab47471f5 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:52.608910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd7570-3b78-490c-a934-573ffe66f990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:52.609549 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd7570-3b78-490c-a934-573ffe66f990 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.609879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd7570-3b78-490c-a934-573ffe66f990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.610079 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd7570-3b78-490c-a934-573ffe66f990 admin admin] PATH_INFO: `/v3/projects` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.610705 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd7570-3b78-490c-a934-573ffe66f990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:52.612602 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd7570-3b78-490c-a934-573ffe66f990 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:52.651682 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd28891-298b-44bc-8ca6-a437febdeda0 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:52.652387 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1416/2846] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:56:52 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:52.656662 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd7570-3b78-490c-a934-573ffe66f990 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:52.701513 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647b4df-ec48-4332-a70f-5caf6b01393e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:52.702133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647b4df-ec48-4332-a70f-5caf6b01393e service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.702408 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647b4df-ec48-4332-a70f-5caf6b01393e service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.702677 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647b4df-ec48-4332-a70f-5caf6b01393e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.715273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:56:52.721123 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd7570-3b78-490c-a934-573ffe66f990 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:52.727205 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22914cc-cbca-476a-81c2-e4187d5b7b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:52.727840 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22914cc-cbca-476a-81c2-e4187d5b7b43 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.728109 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22914cc-cbca-476a-81c2-e4187d5b7b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.728378 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22914cc-cbca-476a-81c2-e4187d5b7b43 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.728995 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22914cc-cbca-476a-81c2-e4187d5b7b43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:52.731148 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22914cc-cbca-476a-81c2-e4187d5b7b43 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:52.757113 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1417/2847] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:56:52 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:56:52.761728 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22914cc-cbca-476a-81c2-e4187d5b7b43 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:52.771834 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22914cc-cbca-476a-81c2-e4187d5b7b43 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:52.773964 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647b4df-ec48-4332-a70f-5caf6b01393e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:52.775642 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647b4df-ec48-4332-a70f-5caf6b01393e service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:52.776728 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b3b8ce-c0e0-4bfc-85fa-7ccb284eeb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:52.777373 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3b8ce-c0e0-4bfc-85fa-7ccb284eeb37 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.777646 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3b8ce-c0e0-4bfc-85fa-7ccb284eeb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.778120 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3b8ce-c0e0-4bfc-85fa-7ccb284eeb37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.778764 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b3b8ce-c0e0-4bfc-85fa-7ccb284eeb37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:52.780342 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b3b8ce-c0e0-4bfc-85fa-7ccb284eeb37 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:52.788777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1431/2848] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:56:52.790930 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1418/2849] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:56:52.795475 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647b4df-ec48-4332-a70f-5caf6b01393e service cinder] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:52.809691 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647b4df-ec48-4332-a70f-5caf6b01393e service cinder] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:52.814974 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd209a-ab08-4cff-9d8b-ec02d596bd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:52.815687 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd209a-ab08-4cff-9d8b-ec02d596bd6f admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.816025 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd209a-ab08-4cff-9d8b-ec02d596bd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.816574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd209a-ab08-4cff-9d8b-ec02d596bd6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.817342 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd209a-ab08-4cff-9d8b-ec02d596bd6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:52.819087 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd209a-ab08-4cff-9d8b-ec02d596bd6f admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:52.830223 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1432/2850] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 05:56:52.834720 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b3b8ce-c0e0-4bfc-85fa-7ccb284eeb37 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:52.844897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b3b8ce-c0e0-4bfc-85fa-7ccb284eeb37 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:52.849722 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a6e09-b173-431b-9267-a44cef584ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:52.850375 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6e09-b173-431b-9267-a44cef584ceb admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.850658 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6e09-b173-431b-9267-a44cef584ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.850936 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6e09-b173-431b-9267-a44cef584ceb admin admin] PATH_INFO: `/v3/projects/f5f58b6fef6a4caaa1c528b0cdfd85c4/users/b28d786fea5042a38ee4547bab77625c/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.866693 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a6e09-b173-431b-9267-a44cef584ceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f58b6fef6a4caaa1c528b0cdfd85c4, user_id=b28d786fea5042a38ee4547bab77625c, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:52.869905 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a6e09-b173-431b-9267-a44cef584ceb admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:52.887661 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1419/2851] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:56:52 2022] PUT /identity/v3/projects/f5f58b6fef6a4caaa1c528b0cdfd85c4/users/b28d786fea5042a38ee4547bab77625c/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:56:52.898548 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec246f-a86e-469b-bf58-80a06435c5ce None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:52.898906 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec246f-a86e-469b-bf58-80a06435c5ce None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:52.899549 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec246f-a86e-469b-bf58-80a06435c5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:52.929528 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-c8ec246f-a86e-469b-bf58-80a06435c5ce None None] MFA Rules not processed for user `b28d786fea5042a38ee4547bab77625c`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:56:52.931339 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c8ec246f-a86e-469b-bf58-80a06435c5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:53.003569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1433/2852] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:56:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:56:53.012605 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a6e09-b173-431b-9267-a44cef584ceb admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:56:53.015464 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9d5a6e09-b173-431b-9267-a44cef584ceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:56:53.101599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a6e09-b173-431b-9267-a44cef584ceb admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:56:53.105711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69978059-53b0-4e84-8775-c058b28fb382 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90c0fabecff2457e931c8c1c36681197', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:56:53.106329 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69978059-53b0-4e84-8775-c058b28fb382 service glance] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:56:53.106607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69978059-53b0-4e84-8775-c058b28fb382 service glance] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:56:53.107051 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69978059-53b0-4e84-8775-c058b28fb382 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:56:53.119625 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69978059-53b0-4e84-8775-c058b28fb382 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:56:53.121518 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69978059-53b0-4e84-8775-c058b28fb382 service glance] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:56:53.135239 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1420/2853] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:56:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:57:02.996505 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=58 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:57:03.128162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=80,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:57:22.467509 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec246f-a86e-469b-bf58-80a06435c5ce None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:57:22.469317 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c8ec246f-a86e-469b-bf58-80a06435c5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:57:22.554147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec246f-a86e-469b-bf58-80a06435c5ce None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:57:22.563077 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03b8fa-bc20-4c91-9180-f2d76f2fbb5e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:57:22.563974 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03b8fa-bc20-4c91-9180-f2d76f2fbb5e service placement] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:57:22.564355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03b8fa-bc20-4c91-9180-f2d76f2fbb5e service placement] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:57:22.564732 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03b8fa-bc20-4c91-9180-f2d76f2fbb5e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:57:22.567871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-cf03b8fa-bc20-4c91-9180-f2d76f2fbb5e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:57:22.584640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03b8fa-bc20-4c91-9180-f2d76f2fbb5e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:57:22.587187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03b8fa-bc20-4c91-9180-f2d76f2fbb5e service placement] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:57:22.598947 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1434/2854] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:57:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:57:32.594129 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=32 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:58:11.440224 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69978059-53b0-4e84-8775-c058b28fb382 service glance] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:11.507641 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69978059-53b0-4e84-8775-c058b28fb382 service glance] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:11.513251 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957928c-2e9d-4ed6-af2a-05be520b5bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:11.513836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957928c-2e9d-4ed6-af2a-05be520b5bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:11.514103 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957928c-2e9d-4ed6-af2a-05be520b5bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:11.514391 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957928c-2e9d-4ed6-af2a-05be520b5bd7 admin admin] PATH_INFO: `/v3/users/b28d786fea5042a38ee4547bab77625c` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:11.515586 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957928c-2e9d-4ed6-af2a-05be520b5bd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b28d786fea5042a38ee4547bab77625c)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:11.517520 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957928c-2e9d-4ed6-af2a-05be520b5bd7 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:11.576811 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8957928c-2e9d-4ed6-af2a-05be520b5bd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b28d786fea5042a38ee4547bab77625c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:11.587179 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8957928c-2e9d-4ed6-af2a-05be520b5bd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b28d786fea5042a38ee4547bab77625c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:11.587968 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8957928c-2e9d-4ed6-af2a-05be520b5bd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b28d786fea5042a38ee4547bab77625c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:11.588915 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8957928c-2e9d-4ed6-af2a-05be520b5bd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b28d786fea5042a38ee4547bab77625c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:11.607842 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-8957928c-2e9d-4ed6-af2a-05be520b5bd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b28d786fea5042a38ee4547bab77625c'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:11.620889 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1421/2855] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:58:11 2022] DELETE /identity/v3/users/b28d786fea5042a38ee4547bab77625c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:58:11.625939 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fa114-6472-4843-96eb-e401e2320522 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:11.626496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fa114-6472-4843-96eb-e401e2320522 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:11.626496 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fa114-6472-4843-96eb-e401e2320522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:11.646504 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-e75fa114-6472-4843-96eb-e401e2320522 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:58:11.647716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-e75fa114-6472-4843-96eb-e401e2320522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:11.712092 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1435/2856] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:58:11.723656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957928c-2e9d-4ed6-af2a-05be520b5bd7 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:11.725125 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8957928c-2e9d-4ed6-af2a-05be520b5bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:11.788786 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957928c-2e9d-4ed6-af2a-05be520b5bd7 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:11.792716 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863c7a5-2bc1-48ad-9a51-10472cea613b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:11.793369 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863c7a5-2bc1-48ad-9a51-10472cea613b service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:11.793649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863c7a5-2bc1-48ad-9a51-10472cea613b service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:11.793989 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863c7a5-2bc1-48ad-9a51-10472cea613b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:11.803377 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863c7a5-2bc1-48ad-9a51-10472cea613b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:11.804799 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863c7a5-2bc1-48ad-9a51-10472cea613b service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:11.815945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1422/2857] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:58:12.105746 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fa114-6472-4843-96eb-e401e2320522 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:12.107465 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-e75fa114-6472-4843-96eb-e401e2320522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:12.121442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fa114-6472-4843-96eb-e401e2320522 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:12.127090 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085f587-2f3a-4f67-8ef3-53ea4218067b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:12.127737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085f587-2f3a-4f67-8ef3-53ea4218067b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:12.128012 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085f587-2f3a-4f67-8ef3-53ea4218067b admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:12.128292 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085f587-2f3a-4f67-8ef3-53ea4218067b admin admin] PATH_INFO: `/v3/projects/f5f58b6fef6a4caaa1c528b0cdfd85c4` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:12.129615 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085f587-2f3a-4f67-8ef3-53ea4218067b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5f58b6fef6a4caaa1c528b0cdfd85c4)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:12.131540 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085f587-2f3a-4f67-8ef3-53ea4218067b admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:12.185312 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6085f587-2f3a-4f67-8ef3-53ea4218067b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5f58b6fef6a4caaa1c528b0cdfd85c4', 'request_id': 'req-6085f587-2f3a-4f67-8ef3-53ea4218067b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:12.191659 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6085f587-2f3a-4f67-8ef3-53ea4218067b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5f58b6fef6a4caaa1c528b0cdfd85c4', 'request_id': 'req-6085f587-2f3a-4f67-8ef3-53ea4218067b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:12.201118 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6085f587-2f3a-4f67-8ef3-53ea4218067b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5f58b6fef6a4caaa1c528b0cdfd85c4', 'request_id': 'req-6085f587-2f3a-4f67-8ef3-53ea4218067b'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:12.211777 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6085f587-2f3a-4f67-8ef3-53ea4218067b admin admin] The token cache is being invalidate because project f5f58b6fef6a4caaa1c528b0cdfd85c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:58:12.212065 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6085f587-2f3a-4f67-8ef3-53ea4218067b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:12.212685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-6085f587-2f3a-4f67-8ef3-53ea4218067b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:12.217253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1436/2858] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:58:12 2022] DELETE /identity/v3/projects/f5f58b6fef6a4caaa1c528b0cdfd85c4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:58:12.219398 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628208c-2903-4ca4-adb8-1ab596116593 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:12.219899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628208c-2903-4ca4-adb8-1ab596116593 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:12.220045 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628208c-2903-4ca4-adb8-1ab596116593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:12.242677 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-8628208c-2903-4ca4-adb8-1ab596116593 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:58:12.243897 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-8628208c-2903-4ca4-adb8-1ab596116593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:12.262656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:58:12.310001 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1423/2859] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:58:12.318251 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484a754-36e4-418e-ac84-37a882eefd14 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:12.318568 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484a754-36e4-418e-ac84-37a882eefd14 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:12.318873 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484a754-36e4-418e-ac84-37a882eefd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:12.339440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-c484a754-36e4-418e-ac84-37a882eefd14 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:58:12.340666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c484a754-36e4-418e-ac84-37a882eefd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:12.350059 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1437/2860] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:58:12.354404 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628208c-2903-4ca4-adb8-1ab596116593 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:12.364545 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628208c-2903-4ca4-adb8-1ab596116593 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:12.369558 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc071af-08df-4422-b0ba-890746351e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:12.370160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc071af-08df-4422-b0ba-890746351e38 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:12.370438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc071af-08df-4422-b0ba-890746351e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:12.370731 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc071af-08df-4422-b0ba-890746351e38 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:12.371380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc071af-08df-4422-b0ba-890746351e38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:58:12.371673 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc071af-08df-4422-b0ba-890746351e38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:12.373363 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc071af-08df-4422-b0ba-890746351e38 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:12.386481 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1424/2861] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:58:12.393247 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484a754-36e4-418e-ac84-37a882eefd14 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:12.402804 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484a754-36e4-418e-ac84-37a882eefd14 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:12.408749 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fef0a-147f-44a1-a2b9-5552570f5962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:12.409397 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fef0a-147f-44a1-a2b9-5552570f5962 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:12.409672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fef0a-147f-44a1-a2b9-5552570f5962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:12.409940 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fef0a-147f-44a1-a2b9-5552570f5962 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:12.410578 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7fef0a-147f-44a1-a2b9-5552570f5962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:12.412465 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7fef0a-147f-44a1-a2b9-5552570f5962 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:12.445737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1438/2862] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:58:12 2022] POST /identity/v3/projects => generated 399 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:58:12.450572 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc071af-08df-4422-b0ba-890746351e38 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:12.466513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc071af-08df-4422-b0ba-890746351e38 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:12.528915 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29b4c6-cab0-4b01-b303-7d58fe2c6852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:12.529554 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29b4c6-cab0-4b01-b303-7d58fe2c6852 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:12.529828 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29b4c6-cab0-4b01-b303-7d58fe2c6852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:12.530120 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29b4c6-cab0-4b01-b303-7d58fe2c6852 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:12.531097 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed29b4c6-cab0-4b01-b303-7d58fe2c6852 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:12.533006 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed29b4c6-cab0-4b01-b303-7d58fe2c6852 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:12.557044 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1425/2863] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:58:12 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:58:12.561672 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fef0a-147f-44a1-a2b9-5552570f5962 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:12.572258 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fef0a-147f-44a1-a2b9-5552570f5962 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:12.577064 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e47d0b-054b-411e-bdb0-4540d64d7965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:12.578050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e47d0b-054b-411e-bdb0-4540d64d7965 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:12.578330 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e47d0b-054b-411e-bdb0-4540d64d7965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:12.578600 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e47d0b-054b-411e-bdb0-4540d64d7965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:12.579235 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e47d0b-054b-411e-bdb0-4540d64d7965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:12.580845 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e47d0b-054b-411e-bdb0-4540d64d7965 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:12.591103 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1439/2864] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:58:12.596238 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29b4c6-cab0-4b01-b303-7d58fe2c6852 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:12.611128 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29b4c6-cab0-4b01-b303-7d58fe2c6852 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:12.616704 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaea9df-687a-4088-befe-767c936761a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:12.617374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaea9df-687a-4088-befe-767c936761a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:12.617655 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaea9df-687a-4088-befe-767c936761a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:12.617930 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaea9df-687a-4088-befe-767c936761a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:12.618579 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaea9df-687a-4088-befe-767c936761a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:12.620214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaea9df-687a-4088-befe-767c936761a0 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:12.632327 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1426/2865] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:58:12.636700 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e47d0b-054b-411e-bdb0-4540d64d7965 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:12.647104 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e47d0b-054b-411e-bdb0-4540d64d7965 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:12.652214 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e050ba-d6e2-4699-8274-d89f8780b82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:12.652848 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e050ba-d6e2-4699-8274-d89f8780b82e admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:12.653173 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e050ba-d6e2-4699-8274-d89f8780b82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:12.653470 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e050ba-d6e2-4699-8274-d89f8780b82e admin admin] PATH_INFO: `/v3/projects/92c22a99d2724bfc97e1e14f14304f8c/users/c2bed19777ff4459ab7f306f2aa9429d/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:12.668359 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e050ba-d6e2-4699-8274-d89f8780b82e admin admin] RBAC: Authorizing `identity:create_grant(project_id=92c22a99d2724bfc97e1e14f14304f8c, user_id=c2bed19777ff4459ab7f306f2aa9429d, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:12.671050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e050ba-d6e2-4699-8274-d89f8780b82e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:12.685622 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1440/2866] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:58:12 2022] PUT /identity/v3/projects/92c22a99d2724bfc97e1e14f14304f8c/users/c2bed19777ff4459ab7f306f2aa9429d/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:58:12.701043 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fd122-98b7-4e1b-882b-7699122365c2 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:12.701393 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fd122-98b7-4e1b-882b-7699122365c2 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:12.701741 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fd122-98b7-4e1b-882b-7699122365c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:12.738008 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-d96fd122-98b7-4e1b-882b-7699122365c2 None None] MFA Rules not processed for user `c2bed19777ff4459ab7f306f2aa9429d`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:58:12.739266 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d96fd122-98b7-4e1b-882b-7699122365c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:12.810974 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1427/2867] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:58:12 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:58:12.826723 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e050ba-d6e2-4699-8274-d89f8780b82e admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:12.830759 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-13e050ba-d6e2-4699-8274-d89f8780b82e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:12.863006 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:58:12.907417 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e050ba-d6e2-4699-8274-d89f8780b82e admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:12.911772 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cee42a-536d-4f9b-8f82-668f7228b0f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:12.912388 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cee42a-536d-4f9b-8f82-668f7228b0f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:12.912666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cee42a-536d-4f9b-8f82-668f7228b0f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:12.912938 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cee42a-536d-4f9b-8f82-668f7228b0f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:12.922052 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cee42a-536d-4f9b-8f82-668f7228b0f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:12.923498 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cee42a-536d-4f9b-8f82-668f7228b0f8 service cinder] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:12.933634 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1441/2868] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:58:12 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:58:17.250392 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fd122-98b7-4e1b-882b-7699122365c2 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:17.252078 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-d96fd122-98b7-4e1b-882b-7699122365c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:17.273811 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:58:17.321738 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fd122-98b7-4e1b-882b-7699122365c2 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:17.326600 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d7258-685a-4383-b1d7-74724e455895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:17.327606 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d7258-685a-4383-b1d7-74724e455895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:17.327928 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d7258-685a-4383-b1d7-74724e455895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:17.328221 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d7258-685a-4383-b1d7-74724e455895 admin admin] PATH_INFO: `/v3/users/2bf60c63ad54497dba7a1b8dbdc5f6a1` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:17.329418 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d7258-685a-4383-b1d7-74724e455895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bf60c63ad54497dba7a1b8dbdc5f6a1)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:17.331349 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d7258-685a-4383-b1d7-74724e455895 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:17.381863 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-273d7258-685a-4383-b1d7-74724e455895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bf60c63ad54497dba7a1b8dbdc5f6a1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:17.390640 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-273d7258-685a-4383-b1d7-74724e455895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bf60c63ad54497dba7a1b8dbdc5f6a1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:17.391443 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-273d7258-685a-4383-b1d7-74724e455895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bf60c63ad54497dba7a1b8dbdc5f6a1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:17.392093 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-273d7258-685a-4383-b1d7-74724e455895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bf60c63ad54497dba7a1b8dbdc5f6a1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:17.405728 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-273d7258-685a-4383-b1d7-74724e455895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bf60c63ad54497dba7a1b8dbdc5f6a1'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:17.419701 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1428/2869] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:58:17 2022] DELETE /identity/v3/users/2bf60c63ad54497dba7a1b8dbdc5f6a1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:58:17.424411 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957fb7a-9e91-42b2-8159-fa7a8c3b6006 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:17.424935 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957fb7a-9e91-42b2-8159-fa7a8c3b6006 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:17.424935 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957fb7a-9e91-42b2-8159-fa7a8c3b6006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:17.446920 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-5957fb7a-9e91-42b2-8159-fa7a8c3b6006 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:58:17.448140 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5957fb7a-9e91-42b2-8159-fa7a8c3b6006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:17.510844 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1442/2870] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:58:17.521041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d7258-685a-4383-b1d7-74724e455895 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:17.522486 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-273d7258-685a-4383-b1d7-74724e455895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:17.591619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d7258-685a-4383-b1d7-74724e455895 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:17.595665 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b89f88b-b2e5-4df9-b29c-b8286349b5c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:17.596406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89f88b-b2e5-4df9-b29c-b8286349b5c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:17.596684 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89f88b-b2e5-4df9-b29c-b8286349b5c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:17.597083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89f88b-b2e5-4df9-b29c-b8286349b5c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:17.608178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b89f88b-b2e5-4df9-b29c-b8286349b5c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:17.609998 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b89f88b-b2e5-4df9-b29c-b8286349b5c9 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:17.620152 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1429/2871] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:58:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:58:17.907104 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957fb7a-9e91-42b2-8159-fa7a8c3b6006 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:17.908551 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-5957fb7a-9e91-42b2-8159-fa7a8c3b6006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:17.921206 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957fb7a-9e91-42b2-8159-fa7a8c3b6006 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:17.926539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbc61c-0149-4c62-a9c6-454d96145bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:17.927185 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbc61c-0149-4c62-a9c6-454d96145bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:17.927464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbc61c-0149-4c62-a9c6-454d96145bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:17.927737 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbc61c-0149-4c62-a9c6-454d96145bde admin admin] PATH_INFO: `/v3/projects/ca0f07bf39e94f60b1aa8524098c89f9` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:17.928959 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbc61c-0149-4c62-a9c6-454d96145bde admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca0f07bf39e94f60b1aa8524098c89f9)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:17.930956 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbc61c-0149-4c62-a9c6-454d96145bde admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:17.988674 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9bfbc61c-0149-4c62-a9c6-454d96145bde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca0f07bf39e94f60b1aa8524098c89f9', 'request_id': 'req-9bfbc61c-0149-4c62-a9c6-454d96145bde'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:17.995508 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9bfbc61c-0149-4c62-a9c6-454d96145bde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca0f07bf39e94f60b1aa8524098c89f9', 'request_id': 'req-9bfbc61c-0149-4c62-a9c6-454d96145bde'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:18.003552 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9bfbc61c-0149-4c62-a9c6-454d96145bde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca0f07bf39e94f60b1aa8524098c89f9', 'request_id': 'req-9bfbc61c-0149-4c62-a9c6-454d96145bde'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:18.014044 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9bfbc61c-0149-4c62-a9c6-454d96145bde admin admin] The token cache is being invalidate because project ca0f07bf39e94f60b1aa8524098c89f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:58:18.014330 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9bfbc61c-0149-4c62-a9c6-454d96145bde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:18.015029 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9bfbc61c-0149-4c62-a9c6-454d96145bde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:18.016685 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1443/2872] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:58:17 2022] DELETE /identity/v3/projects/ca0f07bf39e94f60b1aa8524098c89f9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:58:18.021820 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cde38-2171-45c8-9f94-a5bdfd7ec0df None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:18.022107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cde38-2171-45c8-9f94-a5bdfd7ec0df None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:18.022382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cde38-2171-45c8-9f94-a5bdfd7ec0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:18.041943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-676cde38-2171-45c8-9f94-a5bdfd7ec0df None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:58:18.043149 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-676cde38-2171-45c8-9f94-a5bdfd7ec0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:18.078888 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:58:18.108434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1430/2873] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:58:18.116424 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c990a96-cc65-420f-81f0-de78e98d6abc None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:18.116715 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c990a96-cc65-420f-81f0-de78e98d6abc None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:18.116992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c990a96-cc65-420f-81f0-de78e98d6abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:18.138230 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-7c990a96-cc65-420f-81f0-de78e98d6abc None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:58:18.139475 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-7c990a96-cc65-420f-81f0-de78e98d6abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:18.148616 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1444/2874] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:58:18.153247 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cde38-2171-45c8-9f94-a5bdfd7ec0df None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:18.163062 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cde38-2171-45c8-9f94-a5bdfd7ec0df None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:18.168246 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c49176-d88a-45b8-95e4-83fb8405261d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:18.168841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c49176-d88a-45b8-95e4-83fb8405261d admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:18.169157 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c49176-d88a-45b8-95e4-83fb8405261d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:18.169426 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c49176-d88a-45b8-95e4-83fb8405261d admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:18.170083 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c49176-d88a-45b8-95e4-83fb8405261d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 05:58:18.170382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c49176-d88a-45b8-95e4-83fb8405261d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:18.172082 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c49176-d88a-45b8-95e4-83fb8405261d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:18.184965 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1431/2875] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 05:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 05:58:18.189655 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c990a96-cc65-420f-81f0-de78e98d6abc None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:18.199485 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c990a96-cc65-420f-81f0-de78e98d6abc None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:18.205991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd8a20-c597-4ba2-b8ce-03a9c167d4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:18.206596 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd8a20-c597-4ba2-b8ce-03a9c167d4b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:18.206875 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd8a20-c597-4ba2-b8ce-03a9c167d4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:18.207147 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd8a20-c597-4ba2-b8ce-03a9c167d4b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:18.207799 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd8a20-c597-4ba2-b8ce-03a9c167d4b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:18.209744 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd8a20-c597-4ba2-b8ce-03a9c167d4b4 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:18.239276 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1445/2876] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:58:18 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:58:18.243595 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c49176-d88a-45b8-95e4-83fb8405261d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:18.254428 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c49176-d88a-45b8-95e4-83fb8405261d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:18.315113 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c65e8-8d05-4545-8db5-6f545bfb716a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:18.315716 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c65e8-8d05-4545-8db5-6f545bfb716a admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:18.315997 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c65e8-8d05-4545-8db5-6f545bfb716a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:18.316274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c65e8-8d05-4545-8db5-6f545bfb716a admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:18.316932 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c65e8-8d05-4545-8db5-6f545bfb716a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:18.318934 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c65e8-8d05-4545-8db5-6f545bfb716a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:18.342805 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1432/2877] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:58:18 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:58:18.347501 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd8a20-c597-4ba2-b8ce-03a9c167d4b4 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:18.359398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd8a20-c597-4ba2-b8ce-03a9c167d4b4 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:18.366245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cabe6-bc4e-4e0a-b56b-abaf03663a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:18.366856 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cabe6-bc4e-4e0a-b56b-abaf03663a75 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:18.367133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cabe6-bc4e-4e0a-b56b-abaf03663a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:18.367402 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cabe6-bc4e-4e0a-b56b-abaf03663a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:18.368043 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cabe6-bc4e-4e0a-b56b-abaf03663a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:18.369683 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cabe6-bc4e-4e0a-b56b-abaf03663a75 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:18.379967 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1446/2878] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:58:18.384214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c65e8-8d05-4545-8db5-6f545bfb716a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:18.395016 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c65e8-8d05-4545-8db5-6f545bfb716a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:18.400299 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae924fa-41f4-4373-a0fd-b42f963065bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:18.400898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae924fa-41f4-4373-a0fd-b42f963065bc admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:18.401206 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae924fa-41f4-4373-a0fd-b42f963065bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:18.401488 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae924fa-41f4-4373-a0fd-b42f963065bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:18.402129 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae924fa-41f4-4373-a0fd-b42f963065bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:18.403841 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae924fa-41f4-4373-a0fd-b42f963065bc admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:18.415200 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1433/2879] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:58:18.419650 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cabe6-bc4e-4e0a-b56b-abaf03663a75 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:18.430055 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cabe6-bc4e-4e0a-b56b-abaf03663a75 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:18.435969 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358143a8-e509-4d77-b723-c151e4baa05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:18.436625 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358143a8-e509-4d77-b723-c151e4baa05f admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:18.436905 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358143a8-e509-4d77-b723-c151e4baa05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:18.437161 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358143a8-e509-4d77-b723-c151e4baa05f admin admin] PATH_INFO: `/v3/projects/098526f06e1a43fa9e94e805f72fed14/users/7734565b147c40bebfeea7e093abc393/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:18.439765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:58:18.457936 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358143a8-e509-4d77-b723-c151e4baa05f admin admin] RBAC: Authorizing `identity:create_grant(project_id=098526f06e1a43fa9e94e805f72fed14, user_id=7734565b147c40bebfeea7e093abc393, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:18.460577 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358143a8-e509-4d77-b723-c151e4baa05f admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:18.476508 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1447/2880] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:58:18 2022] PUT /identity/v3/projects/098526f06e1a43fa9e94e805f72fed14/users/7734565b147c40bebfeea7e093abc393/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:58:18.482119 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84cfd2-4a20-45f7-90c4-da5b5cc9265e None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:18.482412 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84cfd2-4a20-45f7-90c4-da5b5cc9265e None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:18.482707 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84cfd2-4a20-45f7-90c4-da5b5cc9265e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:18.507434 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-ab84cfd2-4a20-45f7-90c4-da5b5cc9265e None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:58:18.508711 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ab84cfd2-4a20-45f7-90c4-da5b5cc9265e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:18.569431 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1434/2881] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:58:18.581592 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358143a8-e509-4d77-b723-c151e4baa05f admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:18.583455 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-358143a8-e509-4d77-b723-c151e4baa05f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:18.643323 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358143a8-e509-4d77-b723-c151e4baa05f admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:18.647331 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386745eb-b17c-491e-aa80-48cf91a66ff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:18.647900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386745eb-b17c-491e-aa80-48cf91a66ff4 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:18.648209 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386745eb-b17c-491e-aa80-48cf91a66ff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:18.648473 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386745eb-b17c-491e-aa80-48cf91a66ff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:18.662166 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386745eb-b17c-491e-aa80-48cf91a66ff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:18.663654 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386745eb-b17c-491e-aa80-48cf91a66ff4 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:18.674531 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1448/2882] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 05:58:22.843713 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79837b2f-5bdc-424c-9986-e073870a922c None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:22.844275 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79837b2f-5bdc-424c-9986-e073870a922c None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:22.844275 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79837b2f-5bdc-424c-9986-e073870a922c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:22.884130 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-79837b2f-5bdc-424c-9986-e073870a922c None None] MFA Rules not processed for user `7734565b147c40bebfeea7e093abc393`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:58:22.886034 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-79837b2f-5bdc-424c-9986-e073870a922c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:22.944278 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:58:22.980921 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1435/2883] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:58:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:58:22.985398 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386745eb-b17c-491e-aa80-48cf91a66ff4 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:23.001059 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386745eb-b17c-491e-aa80-48cf91a66ff4 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:23.006580 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248a13b-e0b2-4f66-b9c2-4acd51d8cee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:23.007055 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248a13b-e0b2-4f66-b9c2-4acd51d8cee8 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:23.007335 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248a13b-e0b2-4f66-b9c2-4acd51d8cee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:23.007609 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248a13b-e0b2-4f66-b9c2-4acd51d8cee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:23.008263 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248a13b-e0b2-4f66-b9c2-4acd51d8cee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:23.010274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248a13b-e0b2-4f66-b9c2-4acd51d8cee8 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:23.041950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1449/2884] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 05:58:22 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:58:23.045952 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79837b2f-5bdc-424c-9986-e073870a922c None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:23.058157 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79837b2f-5bdc-424c-9986-e073870a922c None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:23.124487 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e751b-69a4-4a2e-a05b-910c9c3d9f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:23.125297 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e751b-69a4-4a2e-a05b-910c9c3d9f62 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:23.125575 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e751b-69a4-4a2e-a05b-910c9c3d9f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:23.125884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e751b-69a4-4a2e-a05b-910c9c3d9f62 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:23.126519 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e751b-69a4-4a2e-a05b-910c9c3d9f62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:23.128447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e751b-69a4-4a2e-a05b-910c9c3d9f62 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:23.157408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1436/2885] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 05:58:23 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 05:58:23.158786 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248a13b-e0b2-4f66-b9c2-4acd51d8cee8 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:23.177785 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248a13b-e0b2-4f66-b9c2-4acd51d8cee8 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:23.182735 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdafab8-b254-4355-8c77-83d6406e8fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:23.183363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdafab8-b254-4355-8c77-83d6406e8fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:23.185231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdafab8-b254-4355-8c77-83d6406e8fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:23.185231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdafab8-b254-4355-8c77-83d6406e8fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:23.185825 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdafab8-b254-4355-8c77-83d6406e8fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:23.187666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdafab8-b254-4355-8c77-83d6406e8fb3 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:23.199127 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1450/2886] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:58:23.204166 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e751b-69a4-4a2e-a05b-910c9c3d9f62 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:23.215607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e751b-69a4-4a2e-a05b-910c9c3d9f62 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:23.222727 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe60f4-7d71-46a9-ae8d-7e253879292a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:23.223418 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe60f4-7d71-46a9-ae8d-7e253879292a admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:23.223698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe60f4-7d71-46a9-ae8d-7e253879292a admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:23.224078 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe60f4-7d71-46a9-ae8d-7e253879292a admin admin] PATH_INFO: `/v3/projects/a8f42ef66ec8457981fea46df58f1041/users/99d636292cae487cb128157b1f02d314/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:23.247916 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe60f4-7d71-46a9-ae8d-7e253879292a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8f42ef66ec8457981fea46df58f1041, user_id=99d636292cae487cb128157b1f02d314, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:23.252555 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe60f4-7d71-46a9-ae8d-7e253879292a admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:23.282793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1437/2887] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:58:23 2022] PUT /identity/v3/projects/a8f42ef66ec8457981fea46df58f1041/users/99d636292cae487cb128157b1f02d314/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:58:23.284090 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdafab8-b254-4355-8c77-83d6406e8fb3 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:23.297033 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdafab8-b254-4355-8c77-83d6406e8fb3 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:23.374501 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd68761-4257-4e53-b591-24c740aaa48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:23.375561 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd68761-4257-4e53-b591-24c740aaa48c admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:23.375964 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd68761-4257-4e53-b591-24c740aaa48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:23.376354 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd68761-4257-4e53-b591-24c740aaa48c admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:23.377230 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd68761-4257-4e53-b591-24c740aaa48c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:23.378887 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd68761-4257-4e53-b591-24c740aaa48c admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:23.390614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1451/2888] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 05:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 05:58:23.394267 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe60f4-7d71-46a9-ae8d-7e253879292a admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:23.404690 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe60f4-7d71-46a9-ae8d-7e253879292a admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:23.409696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe644ef-9aa3-498e-acac-1a0f99563c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:23.410350 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe644ef-9aa3-498e-acac-1a0f99563c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:23.410624 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe644ef-9aa3-498e-acac-1a0f99563c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:23.410895 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe644ef-9aa3-498e-acac-1a0f99563c5e admin admin] PATH_INFO: `/v3/projects/a8f42ef66ec8457981fea46df58f1041/users/99d636292cae487cb128157b1f02d314/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:23.412833 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe644ef-9aa3-498e-acac-1a0f99563c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8f42ef66ec8457981fea46df58f1041, user_id=99d636292cae487cb128157b1f02d314, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:23.415688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe644ef-9aa3-498e-acac-1a0f99563c5e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:23.426698 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1438/2889] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 05:58:23 2022] PUT /identity/v3/projects/a8f42ef66ec8457981fea46df58f1041/users/99d636292cae487cb128157b1f02d314/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:58:25.875729 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e9322-7b7c-4daa-b147-82439f0105c5 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:25.876222 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e9322-7b7c-4daa-b147-82439f0105c5 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:25.876308 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e9322-7b7c-4daa-b147-82439f0105c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:25.897004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-cf7e9322-7b7c-4daa-b147-82439f0105c5 None None] MFA Rules not processed for user `00928159647d47d6b324c0c11231465e`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:58:25.898301 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-cf7e9322-7b7c-4daa-b147-82439f0105c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:25.977004 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:58:26.025253 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1452/2890] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:58:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:58:26.542547 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe644ef-9aa3-498e-acac-1a0f99563c5e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:26.544361 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1fe644ef-9aa3-498e-acac-1a0f99563c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:26.672922 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe644ef-9aa3-498e-acac-1a0f99563c5e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:26.678746 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe78963-5e24-4df3-bba4-81b532ce6b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:26.679904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe78963-5e24-4df3-bba4-81b532ce6b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:26.680567 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe78963-5e24-4df3-bba4-81b532ce6b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:26.681300 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe78963-5e24-4df3-bba4-81b532ce6b6d admin admin] PATH_INFO: `/v3/users/c2bed19777ff4459ab7f306f2aa9429d` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:26.689424 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe78963-5e24-4df3-bba4-81b532ce6b6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2bed19777ff4459ab7f306f2aa9429d)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:26.692183 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe78963-5e24-4df3-bba4-81b532ce6b6d admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:26.765875 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:58:26.791937 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7fe78963-5e24-4df3-bba4-81b532ce6b6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2bed19777ff4459ab7f306f2aa9429d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:26.810230 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7fe78963-5e24-4df3-bba4-81b532ce6b6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2bed19777ff4459ab7f306f2aa9429d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:26.810935 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7fe78963-5e24-4df3-bba4-81b532ce6b6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2bed19777ff4459ab7f306f2aa9429d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:26.811963 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7fe78963-5e24-4df3-bba4-81b532ce6b6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2bed19777ff4459ab7f306f2aa9429d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:26.836948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-7fe78963-5e24-4df3-bba4-81b532ce6b6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2bed19777ff4459ab7f306f2aa9429d'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:26.873656 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1439/2891] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:58:26 2022] DELETE /identity/v3/users/c2bed19777ff4459ab7f306f2aa9429d => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:58:26.875350 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c1841-1e9a-44d3-b5c3-fae89376b801 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:26.875815 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c1841-1e9a-44d3-b5c3-fae89376b801 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:26.876770 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c1841-1e9a-44d3-b5c3-fae89376b801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:26.922126 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-891c1841-1e9a-44d3-b5c3-fae89376b801 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:58:26.927695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-891c1841-1e9a-44d3-b5c3-fae89376b801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:27.096680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1453/2892] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:58:27.109759 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe78963-5e24-4df3-bba4-81b532ce6b6d admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:27.111433 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7fe78963-5e24-4df3-bba4-81b532ce6b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:27.250389 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe78963-5e24-4df3-bba4-81b532ce6b6d admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:27.266259 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf216bff-34e6-49b9-8f13-9898d3d2a2e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:27.266993 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf216bff-34e6-49b9-8f13-9898d3d2a2e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:27.267283 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf216bff-34e6-49b9-8f13-9898d3d2a2e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:27.267562 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf216bff-34e6-49b9-8f13-9898d3d2a2e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:27.282462 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf216bff-34e6-49b9-8f13-9898d3d2a2e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:27.284495 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf216bff-34e6-49b9-8f13-9898d3d2a2e0 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:27.307368 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1440/2893] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:58:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:58:27.926422 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c1841-1e9a-44d3-b5c3-fae89376b801 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:27.928023 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-891c1841-1e9a-44d3-b5c3-fae89376b801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:27.960156 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c1841-1e9a-44d3-b5c3-fae89376b801 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:27.970124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d41b5b-6438-409d-bdf1-4d790721250e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:27.970991 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d41b5b-6438-409d-bdf1-4d790721250e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:27.971418 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d41b5b-6438-409d-bdf1-4d790721250e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:27.971839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d41b5b-6438-409d-bdf1-4d790721250e admin admin] PATH_INFO: `/v3/projects/92c22a99d2724bfc97e1e14f14304f8c` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:27.973619 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d41b5b-6438-409d-bdf1-4d790721250e admin admin] RBAC: Authorizing `identity:delete_project(project_id=92c22a99d2724bfc97e1e14f14304f8c)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:27.977878 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d41b5b-6438-409d-bdf1-4d790721250e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:28.036502 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-16d41b5b-6438-409d-bdf1-4d790721250e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92c22a99d2724bfc97e1e14f14304f8c', 'request_id': 'req-16d41b5b-6438-409d-bdf1-4d790721250e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:28.043724 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-16d41b5b-6438-409d-bdf1-4d790721250e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92c22a99d2724bfc97e1e14f14304f8c', 'request_id': 'req-16d41b5b-6438-409d-bdf1-4d790721250e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:28.060005 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-16d41b5b-6438-409d-bdf1-4d790721250e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92c22a99d2724bfc97e1e14f14304f8c', 'request_id': 'req-16d41b5b-6438-409d-bdf1-4d790721250e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:28.072884 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-16d41b5b-6438-409d-bdf1-4d790721250e admin admin] The token cache is being invalidate because project 92c22a99d2724bfc97e1e14f14304f8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:58:28.073758 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-16d41b5b-6438-409d-bdf1-4d790721250e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:28.074876 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-16d41b5b-6438-409d-bdf1-4d790721250e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:58:28.098318 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1454/2894] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:58:27 2022] DELETE /identity/v3/projects/92c22a99d2724bfc97e1e14f14304f8c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:58:29.407740 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177c72c-313b-44fd-a89f-18e46a35b033 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:29.408294 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177c72c-313b-44fd-a89f-18e46a35b033 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:29.408559 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177c72c-313b-44fd-a89f-18e46a35b033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:29.441593 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-9177c72c-313b-44fd-a89f-18e46a35b033 None None] MFA Rules not processed for user `99d636292cae487cb128157b1f02d314`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:58:29.442959 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-9177c72c-313b-44fd-a89f-18e46a35b033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:29.541171 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1441/2895] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:58:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 05:58:29.555475 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d41b5b-6438-409d-bdf1-4d790721250e admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:29.557743 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-16d41b5b-6438-409d-bdf1-4d790721250e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:29.652437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d41b5b-6438-409d-bdf1-4d790721250e admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:29.658059 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7288a6-6a85-4190-ba50-bd9ea6926f9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:29.658696 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7288a6-6a85-4190-ba50-bd9ea6926f9c service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:29.659154 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7288a6-6a85-4190-ba50-bd9ea6926f9c service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:29.659421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7288a6-6a85-4190-ba50-bd9ea6926f9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:29.661188 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4a7288a6-6a85-4190-ba50-bd9ea6926f9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:29.675530 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:58:29.731798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7288a6-6a85-4190-ba50-bd9ea6926f9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:29.733352 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7288a6-6a85-4190-ba50-bd9ea6926f9c service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:29.743211 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1455/2896] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:58:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:58:31.094220 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7f3f3-5d2b-43fc-bf64-b1b3d61475ff None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:31.095440 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7f3f3-5d2b-43fc-bf64-b1b3d61475ff None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:31.096005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7f3f3-5d2b-43fc-bf64-b1b3d61475ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:31.124618 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-fea7f3f3-5d2b-43fc-bf64-b1b3d61475ff None None] MFA Rules not processed for user `7734565b147c40bebfeea7e093abc393`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 05:58:31.126236 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-fea7f3f3-5d2b-43fc-bf64-b1b3d61475ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:31.136398 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1442/2897] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 05:58:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 05:58:36.509857 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7288a6-6a85-4190-ba50-bd9ea6926f9c service nova] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:58:36.522374 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7288a6-6a85-4190-ba50-bd9ea6926f9c service nova] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:58:36.529854 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a07b6-f3f6-4858-b5fc-9a49416620f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:58:36.530522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a07b6-f3f6-4858-b5fc-9a49416620f5 service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:58:36.530859 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a07b6-f3f6-4858-b5fc-9a49416620f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:58:36.531177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a07b6-f3f6-4858-b5fc-9a49416620f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:58:36.533367 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f56a07b6-f3f6-4858-b5fc-9a49416620f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:58:36.546807 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a07b6-f3f6-4858-b5fc-9a49416620f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:58:36.548655 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a07b6-f3f6-4858-b5fc-9a49416620f5 service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:58:36.561232 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1456/2898] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:58:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:58:41.114943 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=81,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:58:46.555722 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=27 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:59:06.800058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7f3f3-5d2b-43fc-bf64-b1b3d61475ff None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:59:06.801633 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-fea7f3f3-5d2b-43fc-bf64-b1b3d61475ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:59:06.991396 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7f3f3-5d2b-43fc-bf64-b1b3d61475ff None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:59:06.995342 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653c5fd-d52b-4eee-8963-3b78d53af6f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:59:06.995952 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653c5fd-d52b-4eee-8963-3b78d53af6f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:59:06.996234 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653c5fd-d52b-4eee-8963-3b78d53af6f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:59:06.996506 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653c5fd-d52b-4eee-8963-3b78d53af6f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:59:06.998296 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1653c5fd-d52b-4eee-8963-3b78d53af6f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:59:07.012536 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1653c5fd-d52b-4eee-8963-3b78d53af6f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:59:07.013983 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1653c5fd-d52b-4eee-8963-3b78d53af6f8 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:59:07.025741 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1443/2899] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:59:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:59:11.009319 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a07b6-f3f6-4858-b5fc-9a49416620f5 service nova] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:59:11.011006 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f56a07b6-f3f6-4858-b5fc-9a49416620f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:59:11.023303 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a07b6-f3f6-4858-b5fc-9a49416620f5 service nova] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:59:11.027548 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2174a-d9e8-4326-9014-0a1937ea9bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:59:11.028143 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2174a-d9e8-4326-9014-0a1937ea9bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:59:11.028444 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2174a-d9e8-4326-9014-0a1937ea9bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:59:11.028765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2174a-d9e8-4326-9014-0a1937ea9bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:59:11.030732 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d9c2174a-d9e8-4326-9014-0a1937ea9bc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:59:11.044960 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2174a-d9e8-4326-9014-0a1937ea9bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:59:11.046424 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2174a-d9e8-4326-9014-0a1937ea9bc1 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:59:11.057951 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1457/2900] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:59:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:59:17.020327 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=26 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:59:20.890343 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653c5fd-d52b-4eee-8963-3b78d53af6f8 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:59:20.899881 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653c5fd-d52b-4eee-8963-3b78d53af6f8 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:59:20.903878 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8edb54-7e88-4123-940d-b29d54cf1198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:59:20.904529 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8edb54-7e88-4123-940d-b29d54cf1198 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:59:20.904944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8edb54-7e88-4123-940d-b29d54cf1198 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:59:20.905050 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8edb54-7e88-4123-940d-b29d54cf1198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:59:20.920696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8edb54-7e88-4123-940d-b29d54cf1198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:59:20.922143 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8edb54-7e88-4123-940d-b29d54cf1198 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:59:20.932328 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1444/2901] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:59:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:59:21.052852 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=6 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:59:22.629655 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2174a-d9e8-4326-9014-0a1937ea9bc1 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:59:22.640911 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2174a-d9e8-4326-9014-0a1937ea9bc1 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:59:22.646393 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891adf68-cbac-40a3-be02-257dd9a745da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:59:22.647001 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891adf68-cbac-40a3-be02-257dd9a745da service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:59:22.647269 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891adf68-cbac-40a3-be02-257dd9a745da service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:59:22.647532 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891adf68-cbac-40a3-be02-257dd9a745da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:59:22.649421 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-891adf68-cbac-40a3-be02-257dd9a745da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:59:22.661872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891adf68-cbac-40a3-be02-257dd9a745da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:59:22.663522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891adf68-cbac-40a3-be02-257dd9a745da service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:59:22.675333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1458/2902] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:59:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:59:30.927513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=6 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:59:32.670440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=6 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:59:36.180365 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8edb54-7e88-4123-940d-b29d54cf1198 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:59:36.182191 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7f8edb54-7e88-4123-940d-b29d54cf1198 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:59:36.242333 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891adf68-cbac-40a3-be02-257dd9a745da service nova] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:59:36.243882 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-891adf68-cbac-40a3-be02-257dd9a745da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:59:36.269254 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8edb54-7e88-4123-940d-b29d54cf1198 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:59:36.275038 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee8027-2112-4b5e-b684-48a05a893b01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:59:36.275645 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee8027-2112-4b5e-b684-48a05a893b01 service placement] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:59:36.275911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee8027-2112-4b5e-b684-48a05a893b01 service placement] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:59:36.276181 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee8027-2112-4b5e-b684-48a05a893b01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:59:36.278011 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a1ee8027-2112-4b5e-b684-48a05a893b01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 05:59:36.291869 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee8027-2112-4b5e-b684-48a05a893b01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:59:36.293819 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee8027-2112-4b5e-b684-48a05a893b01 service placement] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:59:36.304041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1445/2903] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 05:59:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 05:59:36.318143 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891adf68-cbac-40a3-be02-257dd9a745da service nova] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:59:36.322981 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca525bd9-6624-4861-9cb1-ca0393693a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:59:36.323614 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca525bd9-6624-4861-9cb1-ca0393693a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:59:36.323909 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca525bd9-6624-4861-9cb1-ca0393693a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:59:36.324181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca525bd9-6624-4861-9cb1-ca0393693a2e admin admin] PATH_INFO: `/v3/users/00928159647d47d6b324c0c11231465e` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:59:36.325329 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca525bd9-6624-4861-9cb1-ca0393693a2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=00928159647d47d6b324c0c11231465e)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:59:36.327751 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca525bd9-6624-4861-9cb1-ca0393693a2e admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:59:36.374985 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-ca525bd9-6624-4861-9cb1-ca0393693a2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00928159647d47d6b324c0c11231465e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:59:36.384030 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-ca525bd9-6624-4861-9cb1-ca0393693a2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00928159647d47d6b324c0c11231465e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:59:36.384640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-ca525bd9-6624-4861-9cb1-ca0393693a2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00928159647d47d6b324c0c11231465e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:59:36.385549 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-ca525bd9-6624-4861-9cb1-ca0393693a2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00928159647d47d6b324c0c11231465e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:59:36.394500 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-ca525bd9-6624-4861-9cb1-ca0393693a2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00928159647d47d6b324c0c11231465e'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:59:36.412716 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1459/2904] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 05:59:36 2022] DELETE /identity/v3/users/00928159647d47d6b324c0c11231465e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 05:59:36.590939 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee8027-2112-4b5e-b684-48a05a893b01 service placement] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 05:59:36.604333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee8027-2112-4b5e-b684-48a05a893b01 service placement] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 05:59:36.663146 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc058e2e-60d1-42cb-b22a-57355593215e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 05:59:36.663777 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc058e2e-60d1-42cb-b22a-57355593215e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 05:59:36.664052 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc058e2e-60d1-42cb-b22a-57355593215e admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 05:59:36.664318 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc058e2e-60d1-42cb-b22a-57355593215e admin admin] PATH_INFO: `/v3/projects/fa25b13404c54cff94e3c12198e9c313` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 05:59:36.665467 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc058e2e-60d1-42cb-b22a-57355593215e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa25b13404c54cff94e3c12198e9c313)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 05:59:36.667350 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc058e2e-60d1-42cb-b22a-57355593215e admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 05:59:36.716252 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fc058e2e-60d1-42cb-b22a-57355593215e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa25b13404c54cff94e3c12198e9c313', 'request_id': 'req-fc058e2e-60d1-42cb-b22a-57355593215e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:59:36.725527 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fc058e2e-60d1-42cb-b22a-57355593215e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa25b13404c54cff94e3c12198e9c313', 'request_id': 'req-fc058e2e-60d1-42cb-b22a-57355593215e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:59:36.735896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fc058e2e-60d1-42cb-b22a-57355593215e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa25b13404c54cff94e3c12198e9c313', 'request_id': 'req-fc058e2e-60d1-42cb-b22a-57355593215e'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:59:36.746754 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fc058e2e-60d1-42cb-b22a-57355593215e admin admin] The token cache is being invalidate because project fa25b13404c54cff94e3c12198e9c313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 05:59:36.747040 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fc058e2e-60d1-42cb-b22a-57355593215e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:59:36.748039 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-fc058e2e-60d1-42cb-b22a-57355593215e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 05:59:36.749630 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1446/2905] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 05:59:36 2022] DELETE /identity/v3/projects/fa25b13404c54cff94e3c12198e9c313 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 05:59:46.409163 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 05:59:46.743472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:00:22.108093 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca525bd9-6624-4861-9cb1-ca0393693a2e admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:00:22.110219 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ca525bd9-6624-4861-9cb1-ca0393693a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:00:22.170876 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca525bd9-6624-4861-9cb1-ca0393693a2e admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:00:22.224370 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0b301-99ce-4202-9008-da72abbfbd9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:00:22.224987 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0b301-99ce-4202-9008-da72abbfbd9e service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:00:22.225299 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0b301-99ce-4202-9008-da72abbfbd9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:00:22.225576 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0b301-99ce-4202-9008-da72abbfbd9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:00:22.227490 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-30b0b301-99ce-4202-9008-da72abbfbd9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:00:22.238561 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0b301-99ce-4202-9008-da72abbfbd9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:00:22.240223 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0b301-99ce-4202-9008-da72abbfbd9e service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:00:22.248910 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1460/2906] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:00:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:00:32.245412 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=46 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:01:56.732957 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc058e2e-60d1-42cb-b22a-57355593215e admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:01:56.735513 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-fc058e2e-60d1-42cb-b22a-57355593215e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:01:56.825493 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc058e2e-60d1-42cb-b22a-57355593215e admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:01:56.831900 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7de1b5-54cf-4bb8-b6b6-0891ecc68e77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:01:56.831900 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7de1b5-54cf-4bb8-b6b6-0891ecc68e77 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:01:56.832139 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7de1b5-54cf-4bb8-b6b6-0891ecc68e77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:01:56.832387 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7de1b5-54cf-4bb8-b6b6-0891ecc68e77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:01:56.834810 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ad7de1b5-54cf-4bb8-b6b6-0891ecc68e77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:01:56.849015 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7de1b5-54cf-4bb8-b6b6-0891ecc68e77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:01:56.850720 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7de1b5-54cf-4bb8-b6b6-0891ecc68e77 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:01:56.861255 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1447/2907] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:01:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:01:56.956439 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0b301-99ce-4202-9008-da72abbfbd9e service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:01:56.957944 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-30b0b301-99ce-4202-9008-da72abbfbd9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:01:57.034461 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0b301-99ce-4202-9008-da72abbfbd9e service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:01:57.038780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052fc36-b249-47b6-8061-4f88df28379b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:01:57.039406 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052fc36-b249-47b6-8061-4f88df28379b service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:01:57.039679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052fc36-b249-47b6-8061-4f88df28379b service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:01:57.039950 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052fc36-b249-47b6-8061-4f88df28379b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:01:57.041923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-f052fc36-b249-47b6-8061-4f88df28379b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:01:57.146709 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052fc36-b249-47b6-8061-4f88df28379b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:01:57.148163 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052fc36-b249-47b6-8061-4f88df28379b service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:01:57.160284 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1461/2908] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:01:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:01:57.234137 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7de1b5-54cf-4bb8-b6b6-0891ecc68e77 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:01:57.235752 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-ad7de1b5-54cf-4bb8-b6b6-0891ecc68e77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:01:57.318896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7de1b5-54cf-4bb8-b6b6-0891ecc68e77 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:01:57.327316 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f96a7a-8b1a-4200-a507-e2fb7a25467f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:01:57.327945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f96a7a-8b1a-4200-a507-e2fb7a25467f service placement] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:01:57.328211 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f96a7a-8b1a-4200-a507-e2fb7a25467f service placement] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:01:57.328479 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f96a7a-8b1a-4200-a507-e2fb7a25467f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:01:57.330351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b1f96a7a-8b1a-4200-a507-e2fb7a25467f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:01:57.343347 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f96a7a-8b1a-4200-a507-e2fb7a25467f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:01:57.344970 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f96a7a-8b1a-4200-a507-e2fb7a25467f service placement] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:01:57.356095 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1448/2909] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:01:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:02:07.155075 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=65 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:02:07.351106 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=60 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:02:26.445073 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052fc36-b249-47b6-8061-4f88df28379b service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:02:26.468340 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052fc36-b249-47b6-8061-4f88df28379b service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:02:26.474169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d41e71-8feb-4119-b405-2c02485ef034 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:02:26.474779 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d41e71-8feb-4119-b405-2c02485ef034 service placement] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:02:26.475061 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d41e71-8feb-4119-b405-2c02485ef034 service placement] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:02:26.475332 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d41e71-8feb-4119-b405-2c02485ef034 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:02:26.477177 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-48d41e71-8feb-4119-b405-2c02485ef034 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:02:26.502128 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d41e71-8feb-4119-b405-2c02485ef034 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:02:26.503630 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d41e71-8feb-4119-b405-2c02485ef034 service placement] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:02:26.513728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1462/2910] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:02:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:02:36.509694 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=15 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:03:44.289366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f96a7a-8b1a-4200-a507-e2fb7a25467f service placement] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:03:44.299667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f96a7a-8b1a-4200-a507-e2fb7a25467f service placement] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:03:44.304427 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc46d3-5010-41cf-8617-77acec426c33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:03:44.305019 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc46d3-5010-41cf-8617-77acec426c33 service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:44.305540 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc46d3-5010-41cf-8617-77acec426c33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:44.305540 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc46d3-5010-41cf-8617-77acec426c33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:44.307700 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b6dc46d3-5010-41cf-8617-77acec426c33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:03:44.372769 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc46d3-5010-41cf-8617-77acec426c33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:03:44.375162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc46d3-5010-41cf-8617-77acec426c33 service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:03:44.390133 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1449/2911] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:03:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:03:45.289124 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d41e71-8feb-4119-b405-2c02485ef034 service placement] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:03:45.290585 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-48d41e71-8feb-4119-b405-2c02485ef034 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:03:45.339960 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d41e71-8feb-4119-b405-2c02485ef034 service placement] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:03:45.346050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752abca-9409-4033-b94c-65673af7200d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:03:45.346517 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752abca-9409-4033-b94c-65673af7200d service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:45.346801 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752abca-9409-4033-b94c-65673af7200d service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:45.347046 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752abca-9409-4033-b94c-65673af7200d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:45.349169 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2752abca-9409-4033-b94c-65673af7200d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:03:45.361700 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752abca-9409-4033-b94c-65673af7200d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:03:45.363497 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752abca-9409-4033-b94c-65673af7200d service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:03:45.377321 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1463/2912] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:03:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:03:49.244197 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc46d3-5010-41cf-8617-77acec426c33 service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:03:49.257398 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc46d3-5010-41cf-8617-77acec426c33 service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:03:49.261248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c8119-4396-46b1-977c-6465df6ee94f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:03:49.261860 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c8119-4396-46b1-977c-6465df6ee94f service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:49.262134 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c8119-4396-46b1-977c-6465df6ee94f service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:49.262382 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c8119-4396-46b1-977c-6465df6ee94f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:49.264203 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-dc5c8119-4396-46b1-977c-6465df6ee94f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:03:49.363249 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c8119-4396-46b1-977c-6465df6ee94f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:03:49.363822 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c8119-4396-46b1-977c-6465df6ee94f service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:03:49.385528 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1450/2913] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:03:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:03:55.370673 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=18 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:03:55.870119 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752abca-9409-4033-b94c-65673af7200d service nova] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:03:55.871788 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2752abca-9409-4033-b94c-65673af7200d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:03:55.887834 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752abca-9409-4033-b94c-65673af7200d service nova] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:03:55.893440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6b35d-ae7f-4751-a8c7-739de1bc97bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:03:55.894103 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6b35d-ae7f-4751-a8c7-739de1bc97bc service placement] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:55.894380 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6b35d-ae7f-4751-a8c7-739de1bc97bc service placement] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:55.894650 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6b35d-ae7f-4751-a8c7-739de1bc97bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:55.898180 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-00e6b35d-ae7f-4751-a8c7-739de1bc97bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:03:55.912990 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6b35d-ae7f-4751-a8c7-739de1bc97bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:03:55.915127 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e6b35d-ae7f-4751-a8c7-739de1bc97bc service placement] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:03:55.929371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1464/2914] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:03:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:03:55.932436 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c8119-4396-46b1-977c-6465df6ee94f service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:03:55.934491 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-dc5c8119-4396-46b1-977c-6465df6ee94f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:03:55.948462 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c8119-4396-46b1-977c-6465df6ee94f service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:03:55.957710 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68c19e-7fa3-4151-9d8b-bc6ec196b4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:03:55.958480 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68c19e-7fa3-4151-9d8b-bc6ec196b4d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:55.958818 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68c19e-7fa3-4151-9d8b-bc6ec196b4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:55.959156 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68c19e-7fa3-4151-9d8b-bc6ec196b4d7 admin admin] PATH_INFO: `/v3/users/7734565b147c40bebfeea7e093abc393` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:55.960354 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68c19e-7fa3-4151-9d8b-bc6ec196b4d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7734565b147c40bebfeea7e093abc393)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:03:55.963473 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68c19e-7fa3-4151-9d8b-bc6ec196b4d7 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:03:56.016366 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3c68c19e-7fa3-4151-9d8b-bc6ec196b4d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7734565b147c40bebfeea7e093abc393'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.023219 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3c68c19e-7fa3-4151-9d8b-bc6ec196b4d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7734565b147c40bebfeea7e093abc393'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.023853 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3c68c19e-7fa3-4151-9d8b-bc6ec196b4d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7734565b147c40bebfeea7e093abc393'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.024437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3c68c19e-7fa3-4151-9d8b-bc6ec196b4d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7734565b147c40bebfeea7e093abc393'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.028753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:03:56.036993 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-3c68c19e-7fa3-4151-9d8b-bc6ec196b4d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7734565b147c40bebfeea7e093abc393'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.048948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1451/2915] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 06:03:55 2022] DELETE /identity/v3/users/7734565b147c40bebfeea7e093abc393 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 06:03:56.230820 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6b35d-ae7f-4751-a8c7-739de1bc97bc service placement] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:03:56.242440 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6b35d-ae7f-4751-a8c7-739de1bc97bc service placement] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:03:56.299541 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0ecb5-075e-4304-bcef-de2f85c85f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:03:56.300161 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0ecb5-075e-4304-bcef-de2f85c85f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:56.300477 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0ecb5-075e-4304-bcef-de2f85c85f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:56.300699 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0ecb5-075e-4304-bcef-de2f85c85f20 admin admin] PATH_INFO: `/v3/projects/098526f06e1a43fa9e94e805f72fed14` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:56.301817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0ecb5-075e-4304-bcef-de2f85c85f20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=098526f06e1a43fa9e94e805f72fed14)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:03:56.303687 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0ecb5-075e-4304-bcef-de2f85c85f20 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:03:56.360761 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-eeb0ecb5-075e-4304-bcef-de2f85c85f20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '098526f06e1a43fa9e94e805f72fed14', 'request_id': 'req-eeb0ecb5-075e-4304-bcef-de2f85c85f20'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.367181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-eeb0ecb5-075e-4304-bcef-de2f85c85f20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '098526f06e1a43fa9e94e805f72fed14', 'request_id': 'req-eeb0ecb5-075e-4304-bcef-de2f85c85f20'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.376729 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-eeb0ecb5-075e-4304-bcef-de2f85c85f20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '098526f06e1a43fa9e94e805f72fed14', 'request_id': 'req-eeb0ecb5-075e-4304-bcef-de2f85c85f20'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.387506 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-eeb0ecb5-075e-4304-bcef-de2f85c85f20 admin admin] The token cache is being invalidate because project 098526f06e1a43fa9e94e805f72fed14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 06:03:56.387798 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-eeb0ecb5-075e-4304-bcef-de2f85c85f20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.388425 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-eeb0ecb5-075e-4304-bcef-de2f85c85f20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.389935 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1465/2916] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 06:03:56 2022] DELETE /identity/v3/projects/098526f06e1a43fa9e94e805f72fed14 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 06:03:56.394311 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68c19e-7fa3-4151-9d8b-bc6ec196b4d7 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:03:56.395889 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-3c68c19e-7fa3-4151-9d8b-bc6ec196b4d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:03:56.409567 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68c19e-7fa3-4151-9d8b-bc6ec196b4d7 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:03:56.489688 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823b8d7-4797-427d-ba82-06381b1cd536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:03:56.490566 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823b8d7-4797-427d-ba82-06381b1cd536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:56.490930 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823b8d7-4797-427d-ba82-06381b1cd536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:56.491300 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823b8d7-4797-427d-ba82-06381b1cd536 admin admin] PATH_INFO: `/v3/users/99d636292cae487cb128157b1f02d314` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:56.492851 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823b8d7-4797-427d-ba82-06381b1cd536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99d636292cae487cb128157b1f02d314)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:03:56.494821 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823b8d7-4797-427d-ba82-06381b1cd536 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:03:56.554210 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a823b8d7-4797-427d-ba82-06381b1cd536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99d636292cae487cb128157b1f02d314'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.563327 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a823b8d7-4797-427d-ba82-06381b1cd536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99d636292cae487cb128157b1f02d314'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.563981 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a823b8d7-4797-427d-ba82-06381b1cd536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99d636292cae487cb128157b1f02d314'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.564589 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a823b8d7-4797-427d-ba82-06381b1cd536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99d636292cae487cb128157b1f02d314'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.577214 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-a823b8d7-4797-427d-ba82-06381b1cd536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99d636292cae487cb128157b1f02d314'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.589011 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1452/2917] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 06:03:56 2022] DELETE /identity/v3/users/99d636292cae487cb128157b1f02d314 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 06:03:56.780429 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0ecb5-075e-4304-bcef-de2f85c85f20 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:03:56.781967 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-eeb0ecb5-075e-4304-bcef-de2f85c85f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:03:56.795518 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0ecb5-075e-4304-bcef-de2f85c85f20 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:03:56.859443 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa04f5b-a258-487f-bfeb-70117f8b9049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:03:56.860094 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa04f5b-a258-487f-bfeb-70117f8b9049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:56.860363 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa04f5b-a258-487f-bfeb-70117f8b9049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:56.860626 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa04f5b-a258-487f-bfeb-70117f8b9049 admin admin] PATH_INFO: `/v3/projects/a8f42ef66ec8457981fea46df58f1041` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:56.862544 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa04f5b-a258-487f-bfeb-70117f8b9049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8f42ef66ec8457981fea46df58f1041)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:03:56.865114 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa04f5b-a258-487f-bfeb-70117f8b9049 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:03:56.897535 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:03:56.934180 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-efa04f5b-a258-487f-bfeb-70117f8b9049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8f42ef66ec8457981fea46df58f1041', 'request_id': 'req-efa04f5b-a258-487f-bfeb-70117f8b9049'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.940570 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-efa04f5b-a258-487f-bfeb-70117f8b9049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8f42ef66ec8457981fea46df58f1041', 'request_id': 'req-efa04f5b-a258-487f-bfeb-70117f8b9049'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.952243 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-efa04f5b-a258-487f-bfeb-70117f8b9049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8f42ef66ec8457981fea46df58f1041', 'request_id': 'req-efa04f5b-a258-487f-bfeb-70117f8b9049'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.963537 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-efa04f5b-a258-487f-bfeb-70117f8b9049 admin admin] The token cache is being invalidate because project a8f42ef66ec8457981fea46df58f1041 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81663) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 06:03:56.963852 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-efa04f5b-a258-487f-bfeb-70117f8b9049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.964613 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-efa04f5b-a258-487f-bfeb-70117f8b9049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:03:56.966165 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1466/2918] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 06:03:56 2022] DELETE /identity/v3/projects/a8f42ef66ec8457981fea46df58f1041 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 06:03:56.971312 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3235fbd-932e-472e-a403-04ae85c1b126 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:56.971831 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3235fbd-932e-472e-a403-04ae85c1b126 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:56.971975 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3235fbd-932e-472e-a403-04ae85c1b126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:57.030721 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-e3235fbd-932e-472e-a403-04ae85c1b126 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 06:03:57.031933 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e3235fbd-932e-472e-a403-04ae85c1b126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:03:57.052170 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:03:57.100088 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1453/2919] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 06:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 06:03:57.109435 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7fb6a-a5b5-4c13-8ca5-c4771c313f1b None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:57.109703 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7fb6a-a5b5-4c13-8ca5-c4771c313f1b None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:57.109988 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7fb6a-a5b5-4c13-8ca5-c4771c313f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:57.132022 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-2da7fb6a-a5b5-4c13-8ca5-c4771c313f1b None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 06:03:57.133569 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2da7fb6a-a5b5-4c13-8ca5-c4771c313f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:03:57.142839 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1467/2920] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 06:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 06:03:57.147458 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3235fbd-932e-472e-a403-04ae85c1b126 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:03:57.157825 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3235fbd-932e-472e-a403-04ae85c1b126 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:03:57.163720 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f62a75-5630-4929-bd18-85da2d717ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:03:57.164356 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f62a75-5630-4929-bd18-85da2d717ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:57.164633 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f62a75-5630-4929-bd18-85da2d717ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:57.164904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f62a75-5630-4929-bd18-85da2d717ea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:57.165599 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f62a75-5630-4929-bd18-85da2d717ea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81664) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 06:03:57.165899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f62a75-5630-4929-bd18-85da2d717ea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:03:57.167500 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f62a75-5630-4929-bd18-85da2d717ea3 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:03:57.179790 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1454/2921] 10.222.0.57 () {58 vars in 1153 bytes} [Mon Sep 26 06:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 06:03:57.186611 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da7fb6a-a5b5-4c13-8ca5-c4771c313f1b None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:03:57.199892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da7fb6a-a5b5-4c13-8ca5-c4771c313f1b None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:03:57.206050 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d052e96-9510-4a6f-8dcb-689400655d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:03:57.206695 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d052e96-9510-4a6f-8dcb-689400655d73 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:57.206973 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d052e96-9510-4a6f-8dcb-689400655d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:57.207234 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d052e96-9510-4a6f-8dcb-689400655d73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:57.207965 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d052e96-9510-4a6f-8dcb-689400655d73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:03:57.210405 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d052e96-9510-4a6f-8dcb-689400655d73 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:03:57.250141 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1468/2922] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 06:03:57 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 06:03:57.254421 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f62a75-5630-4929-bd18-85da2d717ea3 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:03:57.264728 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f62a75-5630-4929-bd18-85da2d717ea3 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:03:57.322576 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6180d-5faf-461e-8ae3-69f5bbea6375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:03:57.323180 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6180d-5faf-461e-8ae3-69f5bbea6375 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:57.323453 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6180d-5faf-461e-8ae3-69f5bbea6375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:57.323713 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6180d-5faf-461e-8ae3-69f5bbea6375 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:57.324350 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6180d-5faf-461e-8ae3-69f5bbea6375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:03:57.326352 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6180d-5faf-461e-8ae3-69f5bbea6375 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:03:57.350455 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1455/2923] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 06:03:57 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 06:03:57.354823 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d052e96-9510-4a6f-8dcb-689400655d73 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:03:57.366270 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d052e96-9510-4a6f-8dcb-689400655d73 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:03:57.371892 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ff984-e62c-4454-aa27-c7b753338fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:03:57.372502 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ff984-e62c-4454-aa27-c7b753338fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:57.372786 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ff984-e62c-4454-aa27-c7b753338fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:57.373053 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ff984-e62c-4454-aa27-c7b753338fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:57.373731 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ff984-e62c-4454-aa27-c7b753338fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:03:57.375351 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ff984-e62c-4454-aa27-c7b753338fc8 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:03:57.386054 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1469/2924] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 06:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 06:03:57.390557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6180d-5faf-461e-8ae3-69f5bbea6375 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:03:57.401380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6180d-5faf-461e-8ae3-69f5bbea6375 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:03:57.407742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa79ad1-79fc-4682-a0bc-69dcca1b4eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:03:57.408333 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa79ad1-79fc-4682-a0bc-69dcca1b4eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:57.408616 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa79ad1-79fc-4682-a0bc-69dcca1b4eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:57.408884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa79ad1-79fc-4682-a0bc-69dcca1b4eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:57.409554 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa79ad1-79fc-4682-a0bc-69dcca1b4eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:03:57.411160 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa79ad1-79fc-4682-a0bc-69dcca1b4eb2 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:03:57.422186 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1456/2925] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 06:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 06:03:57.426430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ff984-e62c-4454-aa27-c7b753338fc8 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:03:57.437126 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ff984-e62c-4454-aa27-c7b753338fc8 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:03:57.442872 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e0caf-b943-493f-b492-f549e0538d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:03:57.443553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e0caf-b943-493f-b492-f549e0538d93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:57.443837 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e0caf-b943-493f-b492-f549e0538d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:57.444152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e0caf-b943-493f-b492-f549e0538d93 admin admin] PATH_INFO: `/v3/projects/fe22d212c0eb4df3800b0e80d722e217/users/ff51bc306c6444ac99ac4c93ea63ac7f/roles/bb454faf5dd643b892c6f9b670ad1f2d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:57.460410 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e0caf-b943-493f-b492-f549e0538d93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe22d212c0eb4df3800b0e80d722e217, user_id=ff51bc306c6444ac99ac4c93ea63ac7f, role_id=bb454faf5dd643b892c6f9b670ad1f2d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:03:57.463071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e0caf-b943-493f-b492-f549e0538d93 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:03:57.478361 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1470/2926] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 06:03:57 2022] PUT /identity/v3/projects/fe22d212c0eb4df3800b0e80d722e217/users/ff51bc306c6444ac99ac4c93ea63ac7f/roles/bb454faf5dd643b892c6f9b670ad1f2d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 06:03:57.483484 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8ed64-4801-4aa3-9f94-07501497cc44 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:57.483770 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8ed64-4801-4aa3-9f94-07501497cc44 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:57.484041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8ed64-4801-4aa3-9f94-07501497cc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:57.503496 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-55e8ed64-4801-4aa3-9f94-07501497cc44 None None] MFA Rules not processed for user `7236c9ce10b9442581cf096a5b4cddf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 06:03:57.504696 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-55e8ed64-4801-4aa3-9f94-07501497cc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:03:57.571334 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1457/2927] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 06:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 06:03:57.583670 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e0caf-b943-493f-b492-f549e0538d93 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:03:57.585660 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-4a7e0caf-b943-493f-b492-f549e0538d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:03:57.662399 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e0caf-b943-493f-b492-f549e0538d93 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:03:57.669113 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a253016-f611-44a4-b42a-f8b24ed07c6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:03:57.670011 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a253016-f611-44a4-b42a-f8b24ed07c6b service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:03:57.670376 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a253016-f611-44a4-b42a-f8b24ed07c6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:03:57.670682 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a253016-f611-44a4-b42a-f8b24ed07c6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:03:57.682063 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a253016-f611-44a4-b42a-f8b24ed07c6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:03:57.683536 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a253016-f611-44a4-b42a-f8b24ed07c6b service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:03:57.693679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1471/2928] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:03:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:04:01.490377 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cc58f-4b46-4508-9a13-50177e304c74 None None] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:04:01.490900 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cc58f-4b46-4508-9a13-50177e304c74 None None] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:04:01.491060 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cc58f-4b46-4508-9a13-50177e304c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:04:01.522438 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.auth.core [None req-929cc58f-4b46-4508-9a13-50177e304c74 None None] MFA Rules not processed for user `ff51bc306c6444ac99ac4c93ea63ac7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81664) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 06:04:01.523680 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-929cc58f-4b46-4508-9a13-50177e304c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:04:01.544660 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:04:01.607086 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1458/2929] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 06:04:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 06:04:01.612245 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a253016-f611-44a4-b42a-f8b24ed07c6b service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:04:01.622780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a253016-f611-44a4-b42a-f8b24ed07c6b service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:04:01.627510 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd09887-0d93-423d-bd59-694640e52dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:04:01.628197 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd09887-0d93-423d-bd59-694640e52dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:04:01.628534 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd09887-0d93-423d-bd59-694640e52dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:04:01.628866 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd09887-0d93-423d-bd59-694640e52dc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:04:01.629668 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd09887-0d93-423d-bd59-694640e52dc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:04:01.631762 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd09887-0d93-423d-bd59-694640e52dc1 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:04:01.663598 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1472/2930] 10.222.0.57 () {60 vars in 1153 bytes} [Mon Sep 26 06:04:01 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 06:04:01.667840 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cc58f-4b46-4508-9a13-50177e304c74 None None] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:04:01.680349 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cc58f-4b46-4508-9a13-50177e304c74 None None] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:04:01.744317 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0728443-d956-47d9-91ef-e1c1e32a7f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:04:01.750603 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0728443-d956-47d9-91ef-e1c1e32a7f36 admin admin] REQUEST_METHOD: `POST` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:04:01.750603 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0728443-d956-47d9-91ef-e1c1e32a7f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:04:01.750603 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0728443-d956-47d9-91ef-e1c1e32a7f36 admin admin] PATH_INFO: `/v3/users` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:04:01.750603 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0728443-d956-47d9-91ef-e1c1e32a7f36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:04:01.750603 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0728443-d956-47d9-91ef-e1c1e32a7f36 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:04:01.771910 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1459/2931] 10.222.0.57 () {60 vars in 1144 bytes} [Mon Sep 26 06:04:01 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 06:04:01.776287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd09887-0d93-423d-bd59-694640e52dc1 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:04:01.787692 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd09887-0d93-423d-bd59-694640e52dc1 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:04:01.793376 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba4cc8-bf86-4339-8154-4f425df78f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:04:01.793997 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba4cc8-bf86-4339-8154-4f425df78f1f admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:04:01.794274 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba4cc8-bf86-4339-8154-4f425df78f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:04:01.794545 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba4cc8-bf86-4339-8154-4f425df78f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:04:01.795185 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba4cc8-bf86-4339-8154-4f425df78f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:04:01.796797 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba4cc8-bf86-4339-8154-4f425df78f1f admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:04:01.809273 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1473/2932] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 06:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 06:04:01.813424 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0728443-d956-47d9-91ef-e1c1e32a7f36 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:04:01.825765 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0728443-d956-47d9-91ef-e1c1e32a7f36 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:04:01.831412 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c466c-54b2-433c-8c3f-bc4bf9f74c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:04:01.832069 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c466c-54b2-433c-8c3f-bc4bf9f74c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:04:01.832337 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c466c-54b2-433c-8c3f-bc4bf9f74c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:04:01.832604 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c466c-54b2-433c-8c3f-bc4bf9f74c04 admin admin] PATH_INFO: `/v3/projects/cddf62ef1f784cf6890bfccb22e5fe33/users/fde12c18f6c14784b10367c46e3daf5d/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:04:01.849975 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c466c-54b2-433c-8c3f-bc4bf9f74c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cddf62ef1f784cf6890bfccb22e5fe33, user_id=fde12c18f6c14784b10367c46e3daf5d, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:04:01.852794 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c466c-54b2-433c-8c3f-bc4bf9f74c04 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:04:01.868468 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1460/2933] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 06:04:01 2022] PUT /identity/v3/projects/cddf62ef1f784cf6890bfccb22e5fe33/users/fde12c18f6c14784b10367c46e3daf5d/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 06:04:01.872578 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba4cc8-bf86-4339-8154-4f425df78f1f admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:04:01.879275 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:04:01.884500 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba4cc8-bf86-4339-8154-4f425df78f1f admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:04:01.955834 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aca989-8718-4423-80e6-406801f999c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:04:01.956457 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aca989-8718-4423-80e6-406801f999c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:04:01.956730 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aca989-8718-4423-80e6-406801f999c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:04:01.956997 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aca989-8718-4423-80e6-406801f999c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:04:01.957679 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aca989-8718-4423-80e6-406801f999c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:04:01.959296 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aca989-8718-4423-80e6-406801f999c3 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:04:01.969666 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1474/2934] 10.222.0.57 () {58 vars in 1122 bytes} [Mon Sep 26 06:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 06:04:01.974107 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c466c-54b2-433c-8c3f-bc4bf9f74c04 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:04:01.985800 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c466c-54b2-433c-8c3f-bc4bf9f74c04 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:04:01.991340 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9a8e2-8833-4ffa-827e-48144e1c045c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:04:01.991590 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9a8e2-8833-4ffa-827e-48144e1c045c admin admin] REQUEST_METHOD: `PUT` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:04:01.991700 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9a8e2-8833-4ffa-827e-48144e1c045c admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:04:01.991976 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9a8e2-8833-4ffa-827e-48144e1c045c admin admin] PATH_INFO: `/v3/projects/cddf62ef1f784cf6890bfccb22e5fe33/users/fde12c18f6c14784b10367c46e3daf5d/roles/4caa7fc301d3427789d438f4c8937d7a` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:04:01.994189 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9a8e2-8833-4ffa-827e-48144e1c045c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cddf62ef1f784cf6890bfccb22e5fe33, user_id=fde12c18f6c14784b10367c46e3daf5d, role_id=4caa7fc301d3427789d438f4c8937d7a)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:04:01.996844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9a8e2-8833-4ffa-827e-48144e1c045c admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:04:02.013178 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1461/2935] 10.222.0.57 () {60 vars in 1483 bytes} [Mon Sep 26 06:04:01 2022] PUT /identity/v3/projects/cddf62ef1f784cf6890bfccb22e5fe33/users/fde12c18f6c14784b10367c46e3daf5d/roles/4caa7fc301d3427789d438f4c8937d7a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 06:04:06.601764 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3454338-564c-46e7-944b-9b69b89259a0 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:04:06.602413 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3454338-564c-46e7-944b-9b69b89259a0 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:04:06.602413 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3454338-564c-46e7-944b-9b69b89259a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:04:06.642407 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-d3454338-564c-46e7-944b-9b69b89259a0 None None] MFA Rules not processed for user `fde12c18f6c14784b10367c46e3daf5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 06:04:06.643871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-d3454338-564c-46e7-944b-9b69b89259a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:04:06.741291 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1475/2936] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 06:04:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 06:04:06.761388 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9a8e2-8833-4ffa-827e-48144e1c045c admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:04:06.764162 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-b9e9a8e2-8833-4ffa-827e-48144e1c045c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:04:06.853176 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9a8e2-8833-4ffa-827e-48144e1c045c admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:04:06.860103 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05384471-b026-46d6-99f1-173da18541aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:04:06.860762 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05384471-b026-46d6-99f1-173da18541aa service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:04:06.861041 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05384471-b026-46d6-99f1-173da18541aa service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:04:06.861344 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05384471-b026-46d6-99f1-173da18541aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:04:06.871111 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:04:06.930778 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05384471-b026-46d6-99f1-173da18541aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:04:06.932658 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05384471-b026-46d6-99f1-173da18541aa service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:04:06.948380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1462/2937] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:04:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:04:16.737255 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=61 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:04:16.938188 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=23 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:04:28.881874 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1746aa6-b848-445a-9fd6-248c2dd813b8 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:04:28.882739 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1746aa6-b848-445a-9fd6-248c2dd813b8 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:04:28.882739 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1746aa6-b848-445a-9fd6-248c2dd813b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:04:28.907680 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-e1746aa6-b848-445a-9fd6-248c2dd813b8 None None] MFA Rules not processed for user `ff51bc306c6444ac99ac4c93ea63ac7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 06:04:28.908885 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-e1746aa6-b848-445a-9fd6-248c2dd813b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:04:28.917805 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1476/2938] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 06:04:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 06:04:28.962408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05384471-b026-46d6-99f1-173da18541aa service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:04:28.964191 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-05384471-b026-46d6-99f1-173da18541aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:04:29.045627 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05384471-b026-46d6-99f1-173da18541aa service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:04:29.049874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a14797-dd9d-4391-b28e-18e55723436f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:04:29.050470 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a14797-dd9d-4391-b28e-18e55723436f service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:04:29.050743 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a14797-dd9d-4391-b28e-18e55723436f service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:04:29.051005 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a14797-dd9d-4391-b28e-18e55723436f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:04:29.052934 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e2a14797-dd9d-4391-b28e-18e55723436f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:04:29.069785 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a14797-dd9d-4391-b28e-18e55723436f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:04:29.072580 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a14797-dd9d-4391-b28e-18e55723436f service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:04:29.084639 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1463/2939] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:04:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:04:29.379689 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1746aa6-b848-445a-9fd6-248c2dd813b8 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:04:29.441838 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1746aa6-b848-445a-9fd6-248c2dd813b8 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:04:29.448632 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee465b24-8529-472b-b208-000120fcda2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:04:29.449563 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee465b24-8529-472b-b208-000120fcda2b service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:04:29.449902 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee465b24-8529-472b-b208-000120fcda2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:04:29.450198 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee465b24-8529-472b-b208-000120fcda2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:04:29.453233 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ee465b24-8529-472b-b208-000120fcda2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:04:29.464621 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee465b24-8529-472b-b208-000120fcda2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:04:29.466088 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee465b24-8529-472b-b208-000120fcda2b service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:04:29.475252 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1477/2940] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:04:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:04:29.960378 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a14797-dd9d-4391-b28e-18e55723436f service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:04:29.961844 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-e2a14797-dd9d-4391-b28e-18e55723436f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:04:29.973308 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a14797-dd9d-4391-b28e-18e55723436f service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:04:29.976960 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd1b8e-74de-4a50-a494-f5e8a25e157b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:04:29.977609 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd1b8e-74de-4a50-a494-f5e8a25e157b service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:04:29.977874 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd1b8e-74de-4a50-a494-f5e8a25e157b service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:04:29.978173 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd1b8e-74de-4a50-a494-f5e8a25e157b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:04:29.979919 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-a8bd1b8e-74de-4a50-a494-f5e8a25e157b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:04:29.993504 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd1b8e-74de-4a50-a494-f5e8a25e157b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:04:29.994939 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd1b8e-74de-4a50-a494-f5e8a25e157b service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:04:30.006031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1464/2941] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:04:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:04:39.470924 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=29 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:04:40.001021 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=41 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:04:44.370459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee465b24-8529-472b-b208-000120fcda2b service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:04:44.385522 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee465b24-8529-472b-b208-000120fcda2b service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:04:44.391021 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bfb20-1d51-44d9-b709-055c67788000 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:04:44.391638 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bfb20-1d51-44d9-b709-055c67788000 service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:04:44.391904 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bfb20-1d51-44d9-b709-055c67788000 service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:04:44.392184 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bfb20-1d51-44d9-b709-055c67788000 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:04:44.393990 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-069bfb20-1d51-44d9-b709-055c67788000 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:04:44.408174 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069bfb20-1d51-44d9-b709-055c67788000 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:04:44.409793 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069bfb20-1d51-44d9-b709-055c67788000 service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:04:44.423756 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1478/2942] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:04:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:04:47.090477 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd1b8e-74de-4a50-a494-f5e8a25e157b service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:04:47.099948 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd1b8e-74de-4a50-a494-f5e8a25e157b service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:04:47.105269 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2059506-c512-4f6c-9c97-8dad631e0b5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:04:47.106085 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2059506-c512-4f6c-9c97-8dad631e0b5c service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:04:47.106360 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2059506-c512-4f6c-9c97-8dad631e0b5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:04:47.106626 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2059506-c512-4f6c-9c97-8dad631e0b5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:04:47.117190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2059506-c512-4f6c-9c97-8dad631e0b5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:04:47.118635 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2059506-c512-4f6c-9c97-8dad631e0b5c service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:04:47.128637 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1465/2943] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:04:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 06:04:49.046575 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bfb20-1d51-44d9-b709-055c67788000 service nova] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:04:49.048059 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-069bfb20-1d51-44d9-b709-055c67788000 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:04:49.062242 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bfb20-1d51-44d9-b709-055c67788000 service nova] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:04:49.067996 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8655db-fbaf-4c0f-96c1-764409bb6f8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:04:49.068594 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8655db-fbaf-4c0f-96c1-764409bb6f8f service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:04:49.068866 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8655db-fbaf-4c0f-96c1-764409bb6f8f service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:04:49.069152 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8655db-fbaf-4c0f-96c1-764409bb6f8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:04:49.071341 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-3f8655db-fbaf-4c0f-96c1-764409bb6f8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:04:49.084130 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8655db-fbaf-4c0f-96c1-764409bb6f8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:04:49.085790 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8655db-fbaf-4c0f-96c1-764409bb6f8f service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:04:49.097071 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1479/2944] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:04:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:04:57.124119 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=6 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:04:59.092054 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=12 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:05:26.121812 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2059506-c512-4f6c-9c97-8dad631e0b5c service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:05:26.131273 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2059506-c512-4f6c-9c97-8dad631e0b5c service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:05:26.136063 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b613ebb-98bd-4c90-9150-b03bd029046d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:05:26.136669 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b613ebb-98bd-4c90-9150-b03bd029046d service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:05:26.136945 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b613ebb-98bd-4c90-9150-b03bd029046d service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:05:26.137249 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b613ebb-98bd-4c90-9150-b03bd029046d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:05:26.139351 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4b613ebb-98bd-4c90-9150-b03bd029046d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:05:26.151129 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b613ebb-98bd-4c90-9150-b03bd029046d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:05:26.152597 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b613ebb-98bd-4c90-9150-b03bd029046d service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:05:26.162615 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1466/2945] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:05:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:05:36.157190 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=6 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:07:07.846077 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca398ea9-5c87-435f-8cd8-a83a68026fa8 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:07:07.846649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca398ea9-5c87-435f-8cd8-a83a68026fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:07:07.846649 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca398ea9-5c87-435f-8cd8-a83a68026fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:07:07.866475 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-ca398ea9-5c87-435f-8cd8-a83a68026fa8 None None] MFA Rules not processed for user `ff51bc306c6444ac99ac4c93ea63ac7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 06:07:07.867682 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-ca398ea9-5c87-435f-8cd8-a83a68026fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:07:07.875145 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1480/2946] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 06:07:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 06:07:08.701898 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b613ebb-98bd-4c90-9150-b03bd029046d service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:07:08.703455 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-4b613ebb-98bd-4c90-9150-b03bd029046d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:07:08.780248 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b613ebb-98bd-4c90-9150-b03bd029046d service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:07:08.785558 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b00e0-1078-48d1-93d2-4bce41046122 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:07:08.786174 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b00e0-1078-48d1-93d2-4bce41046122 service placement] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:07:08.786449 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b00e0-1078-48d1-93d2-4bce41046122 service placement] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:07:08.786723 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b00e0-1078-48d1-93d2-4bce41046122 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:07:08.788624 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1d4b00e0-1078-48d1-93d2-4bce41046122 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:07:08.801836 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b00e0-1078-48d1-93d2-4bce41046122 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:07:08.803419 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b00e0-1078-48d1-93d2-4bce41046122 service placement] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:07:08.814742 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1467/2947] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:07:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:07:17.861507 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=3 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:07:18.809472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=32 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:07:34.433442 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca398ea9-5c87-435f-8cd8-a83a68026fa8 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:07:34.446900 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca398ea9-5c87-435f-8cd8-a83a68026fa8 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:07:34.455763 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0c614-c5b9-43ac-ae74-93243c00ee5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:07:34.456566 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0c614-c5b9-43ac-ae74-93243c00ee5c service placement] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:07:34.456923 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0c614-c5b9-43ac-ae74-93243c00ee5c service placement] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:07:34.457287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0c614-c5b9-43ac-ae74-93243c00ee5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:07:34.459472 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-afd0c614-c5b9-43ac-ae74-93243c00ee5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:07:34.472574 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd0c614-c5b9-43ac-ae74-93243c00ee5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:07:34.474191 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd0c614-c5b9-43ac-ae74-93243c00ee5c service placement] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:07:34.484481 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1481/2948] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:07:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:07:44.480315 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=6 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:09:26.289058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b00e0-1078-48d1-93d2-4bce41046122 service placement] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:09:26.290724 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-1d4b00e0-1078-48d1-93d2-4bce41046122 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:09:26.303311 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b00e0-1078-48d1-93d2-4bce41046122 service placement] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:09:26.307635 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad76576-4363-4274-a3f7-8dedca1e0c71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:09:26.308235 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad76576-4363-4274-a3f7-8dedca1e0c71 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:09:26.308510 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad76576-4363-4274-a3f7-8dedca1e0c71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:09:26.308779 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad76576-4363-4274-a3f7-8dedca1e0c71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:09:26.318444 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad76576-4363-4274-a3f7-8dedca1e0c71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:09:26.319911 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad76576-4363-4274-a3f7-8dedca1e0c71 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:09:26.328797 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1468/2949] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:09:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:09:36.324700 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=6 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:09:37.490554 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958915f-2c87-4ca8-b056-2ffa8f63f102 None None] REQUEST_METHOD: `POST` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:09:37.491168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958915f-2c87-4ca8-b056-2ffa8f63f102 None None] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:09:37.491168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958915f-2c87-4ca8-b056-2ffa8f63f102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:09:37.511992 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.auth.core [None req-e958915f-2c87-4ca8-b056-2ffa8f63f102 None None] MFA Rules not processed for user `ff51bc306c6444ac99ac4c93ea63ac7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81663) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 06:09:37.513373 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-e958915f-2c87-4ca8-b056-2ffa8f63f102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:09:37.521322 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1482/2950] 10.222.0.57 () {58 vars in 958 bytes} [Mon Sep 26 06:09:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 06:09:37.563557 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad76576-4363-4274-a3f7-8dedca1e0c71 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:09:37.572944 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad76576-4363-4274-a3f7-8dedca1e0c71 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:09:37.576964 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf43d8a-672a-4be5-9a0e-61b98f8db499 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46edd6aba77c4404977b1c39e8442b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:09:37.577679 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf43d8a-672a-4be5-9a0e-61b98f8db499 service cinder] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:09:37.577954 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf43d8a-672a-4be5-9a0e-61b98f8db499 service cinder] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:09:37.578226 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf43d8a-672a-4be5-9a0e-61b98f8db499 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:09:37.589617 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf43d8a-672a-4be5-9a0e-61b98f8db499 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:09:37.591058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf43d8a-672a-4be5-9a0e-61b98f8db499 service cinder] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:09:37.602437 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1469/2951] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:09:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:09:37.924868 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958915f-2c87-4ca8-b056-2ffa8f63f102 None None] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:09:37.935768 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958915f-2c87-4ca8-b056-2ffa8f63f102 None None] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:09:37.940342 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb45208-da4b-4b1c-9e30-bdb1c3368484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:09:37.940946 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb45208-da4b-4b1c-9e30-bdb1c3368484 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:09:37.941260 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb45208-da4b-4b1c-9e30-bdb1c3368484 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:09:37.941539 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb45208-da4b-4b1c-9e30-bdb1c3368484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:09:37.949728 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb45208-da4b-4b1c-9e30-bdb1c3368484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:09:37.951168 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb45208-da4b-4b1c-9e30-bdb1c3368484 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:09:37.959469 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1483/2952] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:09:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:09:38.453939 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf43d8a-672a-4be5-9a0e-61b98f8db499 service cinder] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:09:38.464141 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf43d8a-672a-4be5-9a0e-61b98f8db499 service cinder] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:09:38.468474 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c738f97-970e-4c90-a715-7d6d2b4d41ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:09:38.469607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c738f97-970e-4c90-a715-7d6d2b4d41ce service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:09:38.469884 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c738f97-970e-4c90-a715-7d6d2b4d41ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:09:38.470150 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c738f97-970e-4c90-a715-7d6d2b4d41ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:09:38.482408 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c738f97-970e-4c90-a715-7d6d2b4d41ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:09:38.483902 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c738f97-970e-4c90-a715-7d6d2b4d41ce service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:09:38.498406 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1470/2953] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:09:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:09:41.007553 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb45208-da4b-4b1c-9e30-bdb1c3368484 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:09:41.022078 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb45208-da4b-4b1c-9e30-bdb1c3368484 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:09:41.028320 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5b2b5-7a9d-4a45-9acc-440764f7eb6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:09:41.028937 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5b2b5-7a9d-4a45-9acc-440764f7eb6c service nova] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:09:41.029247 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5b2b5-7a9d-4a45-9acc-440764f7eb6c service nova] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:09:41.029765 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5b2b5-7a9d-4a45-9acc-440764f7eb6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:09:41.031754 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-65a5b2b5-7a9d-4a45-9acc-440764f7eb6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:09:41.047863 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5b2b5-7a9d-4a45-9acc-440764f7eb6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:09:41.051459 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5b2b5-7a9d-4a45-9acc-440764f7eb6c service nova] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:09:41.066270 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1484/2954] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:09:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:09:48.491871 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=12 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:09:51.060889 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=15 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:09:51.957094 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c738f97-970e-4c90-a715-7d6d2b4d41ce service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:09:51.968665 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c738f97-970e-4c90-a715-7d6d2b4d41ce service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:09:51.975194 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa88013-04c9-4071-bfb5-602fa56c5e71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92556048cbd04c409c3f4b6fd645627f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:09:51.975788 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa88013-04c9-4071-bfb5-602fa56c5e71 service nova] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:09:51.976065 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa88013-04c9-4071-bfb5-602fa56c5e71 service nova] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:09:51.976380 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa88013-04c9-4071-bfb5-602fa56c5e71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:09:51.986058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa88013-04c9-4071-bfb5-602fa56c5e71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:09:51.987649 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa88013-04c9-4071-bfb5-602fa56c5e71 service nova] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:09:51.998673 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1471/2955] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:09:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:09:55.402591 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5b2b5-7a9d-4a45-9acc-440764f7eb6c service nova] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:09:55.418237 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5b2b5-7a9d-4a45-9acc-440764f7eb6c service nova] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:09:55.424870 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615eff2-e663-4a79-9baa-1bae8f38d314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:09:55.425502 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615eff2-e663-4a79-9baa-1bae8f38d314 service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:09:55.425780 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615eff2-e663-4a79-9baa-1bae8f38d314 service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:09:55.426046 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615eff2-e663-4a79-9baa-1bae8f38d314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:09:55.436191 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615eff2-e663-4a79-9baa-1bae8f38d314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:09:55.437659 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615eff2-e663-4a79-9baa-1bae8f38d314 service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:09:55.447926 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1485/2956] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:09:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:10:01.993877 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=6 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:10:05.443551 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=6 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:10:32.113058 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa88013-04c9-4071-bfb5-602fa56c5e71 service nova] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:10:32.123694 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa88013-04c9-4071-bfb5-602fa56c5e71 service nova] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:10:32.129271 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c70e6f-385b-4aec-8edf-63eb4d59be6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:10:32.129879 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c70e6f-385b-4aec-8edf-63eb4d59be6b service neutron] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:10:32.130159 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c70e6f-385b-4aec-8edf-63eb4d59be6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:10:32.130411 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c70e6f-385b-4aec-8edf-63eb4d59be6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:10:32.139047 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c70e6f-385b-4aec-8edf-63eb4d59be6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:10:32.140466 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c70e6f-385b-4aec-8edf-63eb4d59be6b service neutron] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:10:32.149642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1472/2957] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:10:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:10:42.145607 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=6 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:11:59.433396 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615eff2-e663-4a79-9baa-1bae8f38d314 service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:11:59.469530 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615eff2-e663-4a79-9baa-1bae8f38d314 service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:11:59.473181 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c753544-c1da-48b0-baea-38f401880e7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bf0f594ce7646b7b6ddb8a8553509d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:11:59.473806 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c753544-c1da-48b0-baea-38f401880e7e service neutron] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:11:59.474083 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c753544-c1da-48b0-baea-38f401880e7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:11:59.474355 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c753544-c1da-48b0-baea-38f401880e7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:11:59.560821 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c753544-c1da-48b0-baea-38f401880e7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:11:59.562287 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c753544-c1da-48b0-baea-38f401880e7e service neutron] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:11:59.572257 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1486/2958] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:12:02.698899 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c70e6f-385b-4aec-8edf-63eb4d59be6b service neutron] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:12:02.700374 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-16c70e6f-385b-4aec-8edf-63eb4d59be6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:12:02.744268 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c70e6f-385b-4aec-8edf-63eb4d59be6b service neutron] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:12:02.753054 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2e834-5d6b-4a65-91b9-5b5675ca9431 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:12:02.753713 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e834-5d6b-4a65-91b9-5b5675ca9431 service placement] REQUEST_METHOD: `GET` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:12:02.754140 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e834-5d6b-4a65-91b9-5b5675ca9431 service placement] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:12:02.754274 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e834-5d6b-4a65-91b9-5b5675ca9431 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:12:02.758472 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-7bf2e834-5d6b-4a65-91b9-5b5675ca9431 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:12:02.772447 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2e834-5d6b-4a65-91b9-5b5675ca9431 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:12:02.774105 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2e834-5d6b-4a65-91b9-5b5675ca9431 service placement] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:12:02.785878 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1473/2959] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:12:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:12:06.029671 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c753544-c1da-48b0-baea-38f401880e7e service neutron] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:12:06.039739 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c753544-c1da-48b0-baea-38f401880e7e service neutron] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:12:06.045327 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494f485-a1ca-4c07-b5e2-cfb2c61a2795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:12:06.045960 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494f485-a1ca-4c07-b5e2-cfb2c61a2795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:12:06.046371 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494f485-a1ca-4c07-b5e2-cfb2c61a2795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:12:06.046647 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494f485-a1ca-4c07-b5e2-cfb2c61a2795 admin admin] PATH_INFO: `/v3/users/ff51bc306c6444ac99ac4c93ea63ac7f` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:12:06.047857 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494f485-a1ca-4c07-b5e2-cfb2c61a2795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff51bc306c6444ac99ac4c93ea63ac7f)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:12:06.049817 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494f485-a1ca-4c07-b5e2-cfb2c61a2795 admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:12:06.098468 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c494f485-a1ca-4c07-b5e2-cfb2c61a2795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff51bc306c6444ac99ac4c93ea63ac7f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.106954 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c494f485-a1ca-4c07-b5e2-cfb2c61a2795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff51bc306c6444ac99ac4c93ea63ac7f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.107646 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c494f485-a1ca-4c07-b5e2-cfb2c61a2795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff51bc306c6444ac99ac4c93ea63ac7f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.108437 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c494f485-a1ca-4c07-b5e2-cfb2c61a2795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff51bc306c6444ac99ac4c93ea63ac7f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.118236 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-c494f485-a1ca-4c07-b5e2-cfb2c61a2795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff51bc306c6444ac99ac4c93ea63ac7f'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.136161 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1487/2960] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 06:12:06 2022] DELETE /identity/v3/users/ff51bc306c6444ac99ac4c93ea63ac7f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 06:12:06.313523 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2e834-5d6b-4a65-91b9-5b5675ca9431 service placement] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:12:06.325237 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2e834-5d6b-4a65-91b9-5b5675ca9431 service placement] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:12:06.384310 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:12:06.384867 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:12:06.385167 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:12:06.385439 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57 admin admin] PATH_INFO: `/v3/projects/fe22d212c0eb4df3800b0e80d722e217` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:12:06.386619 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe22d212c0eb4df3800b0e80d722e217)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:12:06.388526 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:12:06.439667 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe22d212c0eb4df3800b0e80d722e217', 'request_id': 'req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.447896 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe22d212c0eb4df3800b0e80d722e217', 'request_id': 'req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.456815 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe22d212c0eb4df3800b0e80d722e217', 'request_id': 'req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.469793 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57 admin admin] The token cache is being invalidate because project fe22d212c0eb4df3800b0e80d722e217 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 06:12:06.470076 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.470690 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.472196 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1474/2961] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 06:12:06 2022] DELETE /identity/v3/projects/fe22d212c0eb4df3800b0e80d722e217 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 06:12:06.476464 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494f485-a1ca-4c07-b5e2-cfb2c61a2795 admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:12:06.477933 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-c494f485-a1ca-4c07-b5e2-cfb2c61a2795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:12:06.491687 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494f485-a1ca-4c07-b5e2-cfb2c61a2795 admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:12:06.509644 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:12:06.551939 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41c410-e486-4542-b4e8-80117989febb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:12:06.552573 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41c410-e486-4542-b4e8-80117989febb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:12:06.552853 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41c410-e486-4542-b4e8-80117989febb admin admin] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:12:06.553119 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41c410-e486-4542-b4e8-80117989febb admin admin] PATH_INFO: `/v3/users/fde12c18f6c14784b10367c46e3daf5d` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:12:06.554376 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41c410-e486-4542-b4e8-80117989febb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fde12c18f6c14784b10367c46e3daf5d)` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:12:06.556304 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41c410-e486-4542-b4e8-80117989febb admin admin] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:12:06.606871 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9b41c410-e486-4542-b4e8-80117989febb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fde12c18f6c14784b10367c46e3daf5d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.615467 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9b41c410-e486-4542-b4e8-80117989febb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fde12c18f6c14784b10367c46e3daf5d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.616070 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9b41c410-e486-4542-b4e8-80117989febb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fde12c18f6c14784b10367c46e3daf5d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.616741 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9b41c410-e486-4542-b4e8-80117989febb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fde12c18f6c14784b10367c46e3daf5d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.623688 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.notifications [None req-9b41c410-e486-4542-b4e8-80117989febb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fde12c18f6c14784b10367c46e3daf5d'} {{(pid=81663) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.640368 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1488/2962] 10.222.0.57 () {58 vars in 1224 bytes} [Mon Sep 26 06:12:06 2022] DELETE /identity/v3/users/fde12c18f6c14784b10367c46e3daf5d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 06:12:06.821969 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57 admin admin] Authenticating user token {{(pid=81664) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:12:06.823952 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.fernet_utils [None req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81664) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:12:06.837031 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94a402-77d8-45c0-bf49-2ddda5d0ac57 admin admin] Validating token access rules against request {{(pid=81664) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:12:06.888450 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:12:06.895360 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af1405-751e-478d-9c46-6900ce5d7049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7236c9ce10b9442581cf096a5b4cddf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb1661fcfa594d16a5b5329ef20333a6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81664) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:12:06.896359 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af1405-751e-478d-9c46-6900ce5d7049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:12:06.896642 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af1405-751e-478d-9c46-6900ce5d7049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:12:06.897315 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af1405-751e-478d-9c46-6900ce5d7049 admin admin] PATH_INFO: `/v3/projects/cddf62ef1f784cf6890bfccb22e5fe33` {{(pid=81664) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:12:06.898539 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af1405-751e-478d-9c46-6900ce5d7049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cddf62ef1f784cf6890bfccb22e5fe33)` {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:12:06.900908 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af1405-751e-478d-9c46-6900ce5d7049 admin admin] RBAC: Authorization granted {{(pid=81664) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:12:06.949993 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d0af1405-751e-478d-9c46-6900ce5d7049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cddf62ef1f784cf6890bfccb22e5fe33', 'request_id': 'req-d0af1405-751e-478d-9c46-6900ce5d7049'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.955909 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d0af1405-751e-478d-9c46-6900ce5d7049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cddf62ef1f784cf6890bfccb22e5fe33', 'request_id': 'req-d0af1405-751e-478d-9c46-6900ce5d7049'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.964673 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d0af1405-751e-478d-9c46-6900ce5d7049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cddf62ef1f784cf6890bfccb22e5fe33', 'request_id': 'req-d0af1405-751e-478d-9c46-6900ce5d7049'} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.975393 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d0af1405-751e-478d-9c46-6900ce5d7049 admin admin] The token cache is being invalidate because project cddf62ef1f784cf6890bfccb22e5fe33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81664) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 06:12:06.975678 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d0af1405-751e-478d-9c46-6900ce5d7049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.976338 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG keystone.notifications [None req-d0af1405-751e-478d-9c46-6900ce5d7049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81664) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 06:12:06.977904 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: [pid: 81664|app: 0|req: 1475/2963] 10.222.0.57 () {58 vars in 1233 bytes} [Mon Sep 26 06:12:06 2022] DELETE /identity/v3/projects/cddf62ef1f784cf6890bfccb22e5fe33 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 06:12:16.637187 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=44,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:12:16.972753 dv-cinder-836083-14-iscsi devstack@keystone.service[81664]: DEBUG dbcounter [-] [81664] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 06:12:41.393430 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41c410-e486-4542-b4e8-80117989febb admin admin] Authenticating user token {{(pid=81663) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 06:12:41.395132 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-9b41c410-e486-4542-b4e8-80117989febb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:12:41.486788 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41c410-e486-4542-b4e8-80117989febb admin admin] Validating token access rules against request {{(pid=81663) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 06:12:41.547968 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3e6e3-dc34-41ab-bfec-c53f94be66f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67fcd65eeb994b2381b83480e030f2ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e44d601292b47b7b66d6f3efa80b450', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81663) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 06:12:41.548604 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3e6e3-dc34-41ab-bfec-c53f94be66f5 service placement] REQUEST_METHOD: `GET` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 06:12:41.548883 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3e6e3-dc34-41ab-bfec-c53f94be66f5 service placement] SCRIPT_NAME: `/identity` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 06:12:41.549231 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3e6e3-dc34-41ab-bfec-c53f94be66f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81663) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 06:12:41.551133 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.fernet_utils [None req-2fa3e6e3-dc34-41ab-bfec-c53f94be66f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81663) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 06:12:41.562596 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3e6e3-dc34-41ab-bfec-c53f94be66f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 06:12:41.564164 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3e6e3-dc34-41ab-bfec-c53f94be66f5 service placement] RBAC: Authorization granted {{(pid=81663) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 06:12:41.574640 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: [pid: 81663|app: 0|req: 1489/2964] 10.222.0.57 () {60 vars in 1365 bytes} [Mon Sep 26 06:12:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 06:12:51.570529 dv-cinder-836083-14-iscsi devstack@keystone.service[81663]: DEBUG dbcounter [-] [81663] Writing DB stats keystone:SELECT=67 {{(pid=81663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}